-- Logs begin at Tue 2022-06-07 07:43:00 UTC, end at Tue 2022-06-07 09:03:43 UTC. -- Jun 07 08:18:30.442464 dv-cinder-740146-15-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 07 08:18:30.450960 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 07 08:18:30.453487 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jun 7 08:18:30 2022] *** Jun 07 08:18:30.453487 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 07 08:18:30.453487 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 07 08:18:30.453487 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: nodename: dv-cinder-740146-15-iscsi Jun 07 08:18:30.453487 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: machine: x86_64 Jun 07 08:18:30.453487 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: clock source: unix Jun 07 08:18:30.453487 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: pcre jit disabled Jun 07 08:18:30.453487 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: detected number of CPU cores: 4 Jun 07 08:18:30.453487 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: current working directory: / Jun 07 08:18:30.453487 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: detected binary path: /usr/bin/uwsgi-core Jun 07 08:18:30.454024 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: your processes number limit is 47680 Jun 07 08:18:30.454024 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: your memory page size is 4096 bytes Jun 07 08:18:30.454024 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: detected max file descriptor number: 2048 Jun 07 08:18:30.454024 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: lock engine: pthread robust mutexes Jun 07 08:18:30.454024 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: thunder lock: enabled Jun 07 08:18:30.454024 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 07 08:18:30.454024 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 07 08:18:30.498271 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: Python main interpreter initialized at 0x55cd580e69c0 Jun 07 08:18:30.498271 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: python threads support enabled Jun 07 08:18:30.498271 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: your server socket listen backlog is limited to 100 connections Jun 07 08:18:30.498271 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: your mercy for graceful operations on workers is 90 seconds Jun 07 08:18:30.498591 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: mapped 403077 bytes (393 KB) for 2 cores Jun 07 08:18:30.498687 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: *** Operational MODE: preforking *** Jun 07 08:18:30.498965 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: *** uWSGI is running in multiple interpreter mode *** Jun 07 08:18:30.498965 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: spawned uWSGI master process (pid: 80344) Jun 07 08:18:30.499158 dv-cinder-740146-15-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 07 08:18:30.500563 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: spawned uWSGI worker 1 (pid: 80357, cores: 1) Jun 07 08:18:30.500563 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: spawned uWSGI worker 2 (pid: 80358, cores: 1) Jun 07 08:18:30.502568 dv-cinder-740146-15-iscsi devstack@keystone.service[80344]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 07 08:18:36.176783 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 08:18:36.176783 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: from cryptography.utils import int_from_bytes Jun 07 08:18:36.177556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 08:18:36.177556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: from cryptography.utils import int_from_bytes Jun 07 08:18:36.178093 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 08:18:36.178093 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: from cryptography.utils import int_from_bytes Jun 07 08:18:36.178544 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 08:18:36.178544 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: from cryptography.utils import int_from_bytes Jun 07 08:18:36.226948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 07 08:18:36.227139 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 07 08:18:36.227331 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] command line args: [] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 07 08:18:36.227602 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 07 08:18:36.227756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 07 08:18:36.227842 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ================================================================================ {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 07 08:18:36.227968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 07 08:18:36.228053 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] admin_token = **** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.228172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] command line args: [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 07 08:18:36.228271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] config_dir = [] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.228445 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 07 08:18:36.228526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.228624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ================================================================================ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 07 08:18:36.228711 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] config_source = [] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.228803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] debug = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.228877 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] admin_token = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.229024 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.229361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] config_dir = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.229361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.229635 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.229780 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] config_source = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.229920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.230048 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] debug = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.230378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.230587 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.230811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.230974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.231117 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] list_limit = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.231245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.231380 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] log_config_append = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.231507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.231632 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.231770 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.231891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] log_dir = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.231891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] log_file = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.232074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.232074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_config_append = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.232256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.232335 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.232412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.232526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_dir = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.232607 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.232683 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_file = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.232683 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.232862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.232862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.233111 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.233193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.233265 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.233378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.233462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.233540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.233662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.233742 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.233818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.233932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.234005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.234123 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.234248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.234324 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.234444 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.234518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.234636 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.234636 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.234832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.234917 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.234988 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.235057 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.235057 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.235265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.235265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] publish_errors = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.235412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.235488 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.235488 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.235625 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.235696 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.235764 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] publish_errors = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.235833 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.235954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.236034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.236108 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.236255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.236409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] standard_threads = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.236482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.236550 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.236714 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.236837 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.236949 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.237100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] standard_threads = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.237221 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] use_journal = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.237334 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.237450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] use_json = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.237450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] use_stderr = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.237665 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.237786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] use_syslog = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.237904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.237904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_journal = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.238126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.238239 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_json = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.238349 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.238467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_stderr = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.238467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_syslog = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.238682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.238796 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 08:18:36.238911 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.239023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.239140 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.239259 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.239370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.239370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.239576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.239576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.239799 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.239799 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.240007 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.240007 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.240221 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.240221 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.240445 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.240445 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.240644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.240760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.240879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.240991 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.241144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.241144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.241362 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.241480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.241591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.241711 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.241899 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.242015 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.242126 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.242126 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.242339 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.242451 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.242573 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.242691 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.242803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.242803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.243017 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.243137 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.243250 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.243250 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.243457 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.243571 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.243683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.243804 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.243916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.243916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.244144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.244256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.244366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.244483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.244596 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.244706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.244822 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.244940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.245079 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.245200 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.245319 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.245431 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.245542 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.245651 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.245745 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.245846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.245942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.246036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.246130 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.246224 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.246329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.246431 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.246526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.246625 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.246726 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.246840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.246941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.247059 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.247203 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.247324 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.247437 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.247550 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.247661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.247783 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.247896 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.248008 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.248119 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.248236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.248354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.248477 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.248590 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.248713 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.248821 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.248937 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.249091 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.249220 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.249335 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.249457 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.249571 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.249682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.249793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.249905 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.250023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.250136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.250249 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.250371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.250485 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.250597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.250715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.250829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.250940 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.251051 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.251169 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.251281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.251401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.251514 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.251625 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.251737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.251848 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.251960 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.252078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.252192 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.252319 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.252461 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.252583 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.252696 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.252820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.252934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.253061 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.253176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.253176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.253396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.253589 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.253716 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.253830 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.253942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.253942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.254151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.254283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.254473 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.254601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.254716 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.254716 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.254926 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.255047 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.255161 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.255273 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.255385 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.255497 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.255615 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.255735 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.255855 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.255968 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.256080 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.256196 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.256322 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.256443 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.256559 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.256678 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.256792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.256904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.257076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.257192 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.257320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.257435 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.257548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.257661 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.257772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.257883 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.258006 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.258120 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.258233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.258351 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.258465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.258576 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.258700 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.258814 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.258927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.259038 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.259150 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.259267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.259388 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.259501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.259619 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.259733 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.259845 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.259956 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.260076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.260190 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.260308 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.260421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.260539 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.260652 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.260776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.260890 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.261083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.261199 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.261311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.261422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.261549 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.261664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.261814 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.261928 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.262039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.262150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.262269 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.262383 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] auth.external = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.262383 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.262592 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.262592 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.262800 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.262922 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.263035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.263035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] auth.password = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.263252 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.263391 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] auth.token = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.263517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.external = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.263517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.263732 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.263732 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.263959 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.264073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.264185 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.264297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.264409 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.password = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.264532 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.264648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.token = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.264760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.264872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.264986 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.265144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.265796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] credential.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.265916 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.266030 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.266143 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.266255 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.266368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.266490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.266605 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.266718 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.266831 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.266943 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.267064 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.267255 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.267372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.267484 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.267597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.267710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.267822 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.267942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.268057 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.268178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.268291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.268291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.268512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.268512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.268734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.268734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.268946 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.269072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.269202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.269325 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.269445 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.269445 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] federation.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.269658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.269771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.269883 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.270004 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.270119 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.270233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.270345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.270458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.270581 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.270696 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.270809 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.270921 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.271034 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.271146 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.271267 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.271382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.271494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.271607 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.271720 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.271832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.271952 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.272068 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.272186 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.272301 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.272413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.272526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.272649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.272764 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.272882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.273021 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.273137 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.273250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.273375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.273375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.273375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.274445 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.274706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.274969 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.275053 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.275243 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.275320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.275512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.275589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.275762 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.275846 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.276009 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.276086 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.276253 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.276336 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.276511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.276596 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.276770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.276847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.277026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.277106 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.277275 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.277365 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.277523 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.277601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.277779 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.277928 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.278035 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.278122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.278279 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.278356 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.278519 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.278597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.278763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.278853 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.279001 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.279079 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.279244 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.279321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.279488 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.279574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.279732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.279810 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.279976 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.280054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.280207 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.280292 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.280448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.280525 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.280693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.280771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.281487 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.281571 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.281749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.281829 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.281985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.282062 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.282220 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.282298 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.282460 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.282537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.282693 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.282771 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.282936 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.283015 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.283178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.283255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.283409 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.283485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.283651 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.283729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.283885 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.283962 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.284126 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.284203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.284369 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.284447 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.284600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.284676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.284832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.284909 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.285102 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.285221 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.285336 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.285449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.285563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.285675 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.285796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.285913 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.286026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.286138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.286251 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.286369 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.286491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.286606 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.286720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.286833 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.286947 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.287060 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.287180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.287303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.287422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.287534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.287648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.287760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.287881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.287995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.288109 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.288222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.288335 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.288449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.288570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.288685 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.288799 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.288912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.289039 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.289155 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.289288 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.289405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.289519 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.289633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.289747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.289860 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.289981 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.290097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.290211 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.290324 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.290437 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.290551 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.290672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.290787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.290900 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.291014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.291127 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.291241 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.291427 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.291545 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.291659 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.291772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.291885 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.291998 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.292120 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.292236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.292350 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.292462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.292575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.292689 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.292810 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.292925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.293052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.293166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.293281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.293394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.293519 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.293634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.293748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.293861 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.293974 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.294087 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.294208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.294325 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.294438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.294551 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.294664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.294777 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.294898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] resource.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.295020 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.295134 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.295248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.295362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.295474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.295595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.295711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.295824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.295938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.296050 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.296164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.296284 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.296401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.296537 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.296651 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.296765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.296886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.297052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.297169 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.297282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] role.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.297395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.297514 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] role.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.297639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.297754 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.297868 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.297981 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.298126 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.298241 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.298363 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.298480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.298593 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] role.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.298706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.298818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] role.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.298931 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.299052 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.299197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.299310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.299423 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.299537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.299650 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.299771 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.299886 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.299999 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.300356 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.300476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.300591 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.300712 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.300827 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.300940 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.301068 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.301183 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.301296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.301421 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.301537 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.301652 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.301765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.301878 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.301992 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.302114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.302230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.302343 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.302456 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.302569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.302683 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.302804 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.302918 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.303031 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.303145 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.303282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.303406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.303521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.303635 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.303749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.303862 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.303976 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.304096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.304212 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.304326 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.304438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.304551 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.304664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.304786 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.304900 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.305036 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.305153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.305266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.305379 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.305508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.305625 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] token.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.305738 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.305852 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.305965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.306078 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.306198 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.306314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.306427 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.306540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.306653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.306765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.306888 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.307003 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.307116 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.307229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.307342 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.307458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.307580 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.307695 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.307808 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.307929 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.308043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.308160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.308282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.308397 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.308511 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.308623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.308736 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.308849 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.308970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.309128 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.309243 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.309356 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.309469 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.309582 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.309706 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.309822 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.connection = **** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.309944 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.310068 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.310183 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.310296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.310409 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.310530 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.310645 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.310758 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.310871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.310983 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.311098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.311218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.311333 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.311522 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.311647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.311762 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.311876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.311989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.312125 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.312247 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.312363 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.312476 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.312589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.312702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.312815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.312936 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.313066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.313182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.313295 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.313408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.313521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.313646 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.313762 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.313875 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.313989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.314102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.314216 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.314337 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 07 08:18:36.314453 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.314566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.314759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.314991 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.315235 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.315474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 08:18:36.315644 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 07 08:18:36.342900 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.343134 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.343971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.344361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.344601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.344828 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.352251 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.353459 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.379848 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.380080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.381355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.381593 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.434357 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.435091 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.483312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.483548 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.484984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.485247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.549836 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.550069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.550551 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.550773 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.550993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.551113 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.551206 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.551354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.551585 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.551811 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.552032 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.552254 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.574684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.574922 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.575024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.575151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.575241 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.575375 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.575457 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.575592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.575673 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.575804 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.575894 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.576016 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.576097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.576230 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.587369 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.588031 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.599937 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.600172 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.600396 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.600476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.600622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.600705 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.600844 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.600925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.601084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.601166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.601302 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.601386 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.601673 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.601921 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 08:18:36.618489 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.618700 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 08:18:36.618783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.619058 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.619140 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 08:18:36.619262 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 08:18:36.619434 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.619516 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.619641 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 08:18:36.619779 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.619915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.620035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.620179 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.620290 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.620447 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.620708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.631634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.631909 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.632172 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.632401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.632483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.632683 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.632957 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.633273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.641355 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.642266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.644626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.644907 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.645216 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.645499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.645580 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.645780 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.645861 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.646038 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.646120 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.646302 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.646384 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.646570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.646651 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.646840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.646922 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.647108 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.647188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.647372 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.647459 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.647662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.647935 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.648202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.673685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.674695 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.675545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.675819 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.676561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.676844 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.681209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.681492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.681770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.682346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.682631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.682914 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.689606 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.689845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.690751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.691033 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.694314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.694600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.694882 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.695159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.695432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.695514 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.695705 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.695868 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.695974 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.696056 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.696321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.696399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.696601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.696876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.697172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.697556 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.715361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.715637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.716639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.716920 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.720233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.721573 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.723996 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.724292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.725341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.725636 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.729316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.730651 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.732906 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.734315 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.736561 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.736860 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.737172 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.737462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.737748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.737981 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.738062 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.738282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.738568 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.738861 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.739142 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.739424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.757422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.757701 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.757971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.758241 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.759048 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.759336 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.759613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.759876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.766666 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.768356 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.769870 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.770151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.771514 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.771794 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.773235 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.773516 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.773792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.774069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.774351 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.774939 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.775225 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.775510 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.775792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.776079 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.787607 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.787895 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.788177 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.788460 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.788748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.789045 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.789332 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.789444 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.789731 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.790015 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.790306 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.790593 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.790876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.791162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.807692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.809802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.810963 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.813637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.814959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.816562 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.817108 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.818226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.818504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.818729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.818865 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.820370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.820651 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.820934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.826079 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.828263 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.828848 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.830494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.831689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.831977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.832259 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.832533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.833876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.834168 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.834450 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.834741 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.841580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.841870 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.842146 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.843809 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.844098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.844381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.850142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.850423 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.850704 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.850980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.851251 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.851528 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.851798 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.852069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.852348 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.852481 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 08:18:36.852611 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.852742 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.852870 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.853046 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.853217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.853344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.853473 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.853601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.853738 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.853875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.854003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.854238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.854514 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.854782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.855052 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.855329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.855689 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.855966 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 08:18:36.892074 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cd580e69c0 pid: 80357 (default app) Jun 07 08:18:36.894507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cd580e69c0 pid: 80358 (default app) Jun 07 08:18:36.895478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c26a4-d398-4df7-9e58-abe2172fc04d None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:36.895830 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c26a4-d398-4df7-9e58-abe2172fc04d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:36.896113 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c26a4-d398-4df7-9e58-abe2172fc04d None None] PATH_INFO: `/v3` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:36.900299 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Tue Jun 7 08:18:36 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:18:45.534455 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9b6a2-bb58-4267-b592-d65343d7c14a None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:45.534991 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9b6a2-bb58-4267-b592-d65343d7c14a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:45.534991 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9b6a2-bb58-4267-b592-d65343d7c14a None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:45.539279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:18:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:18:45.544738 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94720178-e1fe-4b59-aeec-6bb48a0580e1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:45.545199 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94720178-e1fe-4b59-aeec-6bb48a0580e1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:45.545334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94720178-e1fe-4b59-aeec-6bb48a0580e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:45.612836 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: INFO dbcounter [None req-94720178-e1fe-4b59-aeec-6bb48a0580e1 None None] Registered counter for database keystone Jun 07 08:18:45.634172 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG oslo_db.sqlalchemy.engines [None req-94720178-e1fe-4b59-aeec-6bb48a0580e1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80357) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 07 08:18:45.641084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writer thread running {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 07 08:18:45.917337 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG passlib.handlers.bcrypt [None req-94720178-e1fe-4b59-aeec-6bb48a0580e1 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80357) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 07 08:18:45.917760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG passlib.handlers.bcrypt [None req-94720178-e1fe-4b59-aeec-6bb48a0580e1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80357) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 07 08:18:45.974358 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-94720178-e1fe-4b59-aeec-6bb48a0580e1 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:18:45.975984 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-94720178-e1fe-4b59-aeec-6bb48a0580e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:18:46.098287 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:18:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 554 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 07 08:18:46.295835 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9b6a2-bb58-4267-b592-d65343d7c14a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:46.364698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: INFO dbcounter [None req-0aa9b6a2-bb58-4267-b592-d65343d7c14a None None] Registered counter for database keystone Jun 07 08:18:46.384265 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG oslo_db.sqlalchemy.engines [None req-0aa9b6a2-bb58-4267-b592-d65343d7c14a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80358) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 07 08:18:46.390415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writer thread running {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 07 08:18:46.611434 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9b6a2-bb58-4267-b592-d65343d7c14a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:46.615637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabdef1-d6a0-4630-9aaa-eec1b63f845d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:46.616293 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabdef1-d6a0-4630-9aaa-eec1b63f845d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:46.616577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabdef1-d6a0-4630-9aaa-eec1b63f845d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:46.616852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabdef1-d6a0-4630-9aaa-eec1b63f845d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:46.625355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabdef1-d6a0-4630-9aaa-eec1b63f845d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:46.677221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ebabdef1-d6a0-4630-9aaa-eec1b63f845d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 07 08:18:46.677563 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabdef1-d6a0-4630-9aaa-eec1b63f845d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:46.686785 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Tue Jun 7 08:18:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:18:46.693271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94720178-e1fe-4b59-aeec-6bb48a0580e1 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:46.707340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94720178-e1fe-4b59-aeec-6bb48a0580e1 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:46.711634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220d5ee-3b87-4e2b-bba7-43d9c4304cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:46.712752 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220d5ee-3b87-4e2b-bba7-43d9c4304cec admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:46.713057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220d5ee-3b87-4e2b-bba7-43d9c4304cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:46.713335 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220d5ee-3b87-4e2b-bba7-43d9c4304cec admin admin] PATH_INFO: `/v3/projects/df1e3ab9af2d4109a302f1229baa02a8` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:46.714829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220d5ee-3b87-4e2b-bba7-43d9c4304cec admin admin] RBAC: Authorizing `identity:get_project(project_id=df1e3ab9af2d4109a302f1229baa02a8)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:46.766270 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7220d5ee-3b87-4e2b-bba7-43d9c4304cec admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 07 08:18:46.766611 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220d5ee-3b87-4e2b-bba7-43d9c4304cec admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:46.772331 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Tue Jun 7 08:18:46 2022] GET /identity/v3/projects/df1e3ab9af2d4109a302f1229baa02a8 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:18:48.945291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e7639-d53d-4bd9-aac4-88f0443309ba None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:48.945835 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e7639-d53d-4bd9-aac4-88f0443309ba None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:48.945835 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e7639-d53d-4bd9-aac4-88f0443309ba None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:48.949912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:18:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:18:48.955774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b9c20-ab56-43e8-8cdd-7bf4c7c24976 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:48.956192 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b9c20-ab56-43e8-8cdd-7bf4c7c24976 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:48.956339 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b9c20-ab56-43e8-8cdd-7bf4c7c24976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:48.975525 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-431b9c20-ab56-43e8-8cdd-7bf4c7c24976 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:18:48.977394 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-431b9c20-ab56-43e8-8cdd-7bf4c7c24976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:18:48.985591 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:18:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 07 08:18:49.007623 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e7639-d53d-4bd9-aac4-88f0443309ba None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:49.020271 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e7639-d53d-4bd9-aac4-88f0443309ba None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:49.024423 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a9a57-83ee-408b-9002-01788d9f727e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:49.025079 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a9a57-83ee-408b-9002-01788d9f727e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:49.025366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a9a57-83ee-408b-9002-01788d9f727e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:49.025645 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a9a57-83ee-408b-9002-01788d9f727e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:49.033881 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a9a57-83ee-408b-9002-01788d9f727e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:49.035656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a9a57-83ee-408b-9002-01788d9f727e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:49.044267 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Tue Jun 7 08:18:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:18:49.049312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b9c20-ab56-43e8-8cdd-7bf4c7c24976 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:49.058449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b9c20-ab56-43e8-8cdd-7bf4c7c24976 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:49.062923 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750a998-3b74-4525-8cfd-52e59e49b1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:49.063796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750a998-3b74-4525-8cfd-52e59e49b1bf admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:49.064081 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750a998-3b74-4525-8cfd-52e59e49b1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:49.064360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750a998-3b74-4525-8cfd-52e59e49b1bf admin admin] PATH_INFO: `/v3/users/4ce5a782a83f41fbb386c88e6d77ba55` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:49.065769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750a998-3b74-4525-8cfd-52e59e49b1bf admin admin] RBAC: Authorizing `identity:get_user(user_id=4ce5a782a83f41fbb386c88e6d77ba55)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:49.067593 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750a998-3b74-4525-8cfd-52e59e49b1bf admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:49.069163 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Tue Jun 7 08:18:49 2022] GET /identity/v3/users/4ce5a782a83f41fbb386c88e6d77ba55 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:18:51.341484 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51594765-97ff-409f-bf3e-d469eba7cb58 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:51.342001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51594765-97ff-409f-bf3e-d469eba7cb58 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:51.342095 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51594765-97ff-409f-bf3e-d469eba7cb58 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:51.343938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:18:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:18:51.349179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a39366-1def-4d89-ad11-00494c693a14 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:51.349611 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a39366-1def-4d89-ad11-00494c693a14 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:51.349869 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a39366-1def-4d89-ad11-00494c693a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:51.366441 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b6a39366-1def-4d89-ad11-00494c693a14 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:18:51.367616 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b6a39366-1def-4d89-ad11-00494c693a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:18:51.427772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cf51e-9406-47c7-a710-5b6106729e1c None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:51.428065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cf51e-9406-47c7-a710-5b6106729e1c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:51.428350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cf51e-9406-47c7-a710-5b6106729e1c None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:51.429734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:18:51.434700 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809d9f3-88e8-4b0f-93db-a25ce0af83a8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:51.435048 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809d9f3-88e8-4b0f-93db-a25ce0af83a8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:51.435389 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809d9f3-88e8-4b0f-93db-a25ce0af83a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:51.447413 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:18:51.471107 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a39366-1def-4d89-ad11-00494c693a14 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:51.474889 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG passlib.handlers.bcrypt [None req-0809d9f3-88e8-4b0f-93db-a25ce0af83a8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80358) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 07 08:18:51.475313 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG passlib.handlers.bcrypt [None req-0809d9f3-88e8-4b0f-93db-a25ce0af83a8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80358) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 07 08:18:51.493868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:51.522113 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0809d9f3-88e8-4b0f-93db-a25ce0af83a8 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:18:51.527187 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0809d9f3-88e8-4b0f-93db-a25ce0af83a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:18:51.573982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a39366-1def-4d89-ad11-00494c693a14 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:51.606669 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88fdf4-5e6f-4b93-8801-4d1daf1f7b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:51.607458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88fdf4-5e6f-4b93-8801-4d1daf1f7b45 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:51.607738 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88fdf4-5e6f-4b93-8801-4d1daf1f7b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:51.608014 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88fdf4-5e6f-4b93-8801-4d1daf1f7b45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:51.619344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf88fdf4-5e6f-4b93-8801-4d1daf1f7b45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:18:51.621316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:18:51.631032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-cf88fdf4-5e6f-4b93-8801-4d1daf1f7b45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:18:51.631032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:18:51.631032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:18:51.631032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:18:51.631032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:18:51.631032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:18:51.631032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:18:51.631032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:18:51.631032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:18:51.631032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:18:51.631032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:18:51.631032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:18:51.631032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:18:51.631032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:18:51.631032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:18:51.631032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:18:51.632095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:18:51.632095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:18:51.632095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:18:51.632095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:18:51.632095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:18:51.632095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:18:51.632095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:18:51.632095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:18:51.632095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:18:51.632095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:18:51.632095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:18:51.632095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:18:51.632095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:18:51.632095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:18:51.632095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:18:51.632095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:18:51.632898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:18:51.632898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:18:51.632898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:18:51.632898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:18:51.632898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:18:51.632898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:18:51.632898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:18:51.632898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:18:51.632898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:18:51.632898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:18:51.632898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:18:51.632898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:18:51.632898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:18:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:18:51.637857 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0809d9f3-88e8-4b0f-93db-a25ce0af83a8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:51.644454 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88fdf4-5e6f-4b93-8801-4d1daf1f7b45 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:51.723360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:51.736318 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0809d9f3-88e8-4b0f-93db-a25ce0af83a8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:51.738611 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:51.740684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88fdf4-5e6f-4b93-8801-4d1daf1f7b45 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:51.773758 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ff4b1-889b-4a46-b42d-244f90ac1712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:51.774668 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ff4b1-889b-4a46-b42d-244f90ac1712 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:51.774784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8cbd0c-5338-4d29-b74c-d90130c50f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:51.774948 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ff4b1-889b-4a46-b42d-244f90ac1712 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:51.775219 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ff4b1-889b-4a46-b42d-244f90ac1712 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:51.775412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cbd0c-5338-4d29-b74c-d90130c50f36 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:51.775692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cbd0c-5338-4d29-b74c-d90130c50f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:51.775965 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cbd0c-5338-4d29-b74c-d90130c50f36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:51.776132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ff4b1-889b-4a46-b42d-244f90ac1712 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:18:51.776437 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ff4b1-889b-4a46-b42d-244f90ac1712 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:51.777907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ff4b1-889b-4a46-b42d-244f90ac1712 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:51.790438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:18:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:18:51.796014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ff4b1-889b-4a46-b42d-244f90ac1712 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:51.842848 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8cbd0c-5338-4d29-b74c-d90130c50f36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:51.844351 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8cbd0c-5338-4d29-b74c-d90130c50f36 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:51.893760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ff4b1-889b-4a46-b42d-244f90ac1712 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:51.924354 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:51.926935 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6fff4a-07a9-409b-93fa-e1a6ad0ccd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:51.927807 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fff4a-07a9-409b-93fa-e1a6ad0ccd17 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:51.928086 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fff4a-07a9-409b-93fa-e1a6ad0ccd17 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:51.928358 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fff4a-07a9-409b-93fa-e1a6ad0ccd17 None admin] PATH_INFO: `/v3/users/4ce5a782a83f41fbb386c88e6d77ba55` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:51.929908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6fff4a-07a9-409b-93fa-e1a6ad0ccd17 None admin] RBAC: Authorizing `identity:get_user(user_id=4ce5a782a83f41fbb386c88e6d77ba55)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:51.931354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6fff4a-07a9-409b-93fa-e1a6ad0ccd17 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:51.933025 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 9/16] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:18:51 2022] GET /identity/v3/users/4ce5a782a83f41fbb386c88e6d77ba55 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:18:51.938815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6fff4a-07a9-409b-93fa-e1a6ad0ccd17 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:51.944580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 8/17] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:18:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 08:18:51.950017 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8cbd0c-5338-4d29-b74c-d90130c50f36 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:51.974253 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:52.038305 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6fff4a-07a9-409b-93fa-e1a6ad0ccd17 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:52.044919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8cbd0c-5338-4d29-b74c-d90130c50f36 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:52.072250 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34edf7f1-bf8c-485e-b225-3cf49633074a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:52.073071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edf7f1-bf8c-485e-b225-3cf49633074a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:52.073357 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edf7f1-bf8c-485e-b225-3cf49633074a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:52.073635 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edf7f1-bf8c-485e-b225-3cf49633074a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:52.074805 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34edf7f1-bf8c-485e-b225-3cf49633074a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:52.076249 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34edf7f1-bf8c-485e-b225-3cf49633074a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:52.077893 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 10/18] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:18:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:18:52.078753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf48cf2-18a4-4b77-bfb7-2a2db9a7228c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:52.079539 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf48cf2-18a4-4b77-bfb7-2a2db9a7228c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:52.079822 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf48cf2-18a4-4b77-bfb7-2a2db9a7228c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:52.080096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf48cf2-18a4-4b77-bfb7-2a2db9a7228c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:52.083473 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34edf7f1-bf8c-485e-b225-3cf49633074a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:52.090720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf48cf2-18a4-4b77-bfb7-2a2db9a7228c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:52.092164 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf48cf2-18a4-4b77-bfb7-2a2db9a7228c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:52.110873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-edf48cf2-18a4-4b77-bfb7-2a2db9a7228c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:18:52.110873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:18:52.110873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:18:52.110873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:18:52.110873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:18:52.110873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:18:52.110873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:18:52.110873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:18:52.110873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:18:52.110873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:18:52.110873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:18:52.110873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:18:52.110873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:18:52.110873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:18:52.110873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:18:52.111823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:18:52.111823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:18:52.111823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:18:52.111823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:18:52.111823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:18:52.111823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:18:52.111823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:18:52.111823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:18:52.111823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:18:52.111823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:18:52.111823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:18:52.111823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:18:52.111823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:18:52.111823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:18:52.111823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:18:52.111823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:18:52.111823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:18:52.112782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:18:52.112782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:18:52.112782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:18:52.112782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:18:52.112782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 9/19] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:18:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:18:52.117726 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf48cf2-18a4-4b77-bfb7-2a2db9a7228c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:52.150783 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:52.181671 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34edf7f1-bf8c-485e-b225-3cf49633074a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:52.183164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:52.217320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3f75c-fc7c-4dc3-bcc3-891403ce8aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:52.217982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3f75c-fc7c-4dc3-bcc3-891403ce8aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:52.218778 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3f75c-fc7c-4dc3-bcc3-891403ce8aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:52.218874 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf48cf2-18a4-4b77-bfb7-2a2db9a7228c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:52.219058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3f75c-fc7c-4dc3-bcc3-891403ce8aa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:52.219891 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3f75c-fc7c-4dc3-bcc3-891403ce8aa9 None admin] RBAC: Adding query filter params (role.id=8c63c00418a346b29f7db3f87a404e70, scope.domain.id=default, user.id=4ce5a782a83f41fbb386c88e6d77ba55) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:18:52.220207 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3f75c-fc7c-4dc3-bcc3-891403ce8aa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:52.221678 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3f75c-fc7c-4dc3-bcc3-891403ce8aa9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:52.229877 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 11/20] 10.222.0.17 () {56 vars in 1322 bytes} [Tue Jun 7 08:18:52 2022] GET /identity/v3/role_assignments?user.id=4ce5a782a83f41fbb386c88e6d77ba55&scope.domain.id=default&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:18:52.253483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5868f88-d558-49bb-96e6-42c89f100a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:52.254163 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5868f88-d558-49bb-96e6-42c89f100a09 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:52.254440 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5868f88-d558-49bb-96e6-42c89f100a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:52.254715 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5868f88-d558-49bb-96e6-42c89f100a09 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:52.255459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5868f88-d558-49bb-96e6-42c89f100a09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:18:52.255765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5868f88-d558-49bb-96e6-42c89f100a09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:52.257274 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5868f88-d558-49bb-96e6-42c89f100a09 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:52.276417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 10/21] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:18:54.709393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85765276-dfb6-4f0a-a2e7-b488ad739592 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:54.709901 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85765276-dfb6-4f0a-a2e7-b488ad739592 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:54.709997 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85765276-dfb6-4f0a-a2e7-b488ad739592 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:54.711309 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 12/22] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:18:54.716483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70547b27-ab43-4ade-a9be-7662802e73cd None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:54.716879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70547b27-ab43-4ade-a9be-7662802e73cd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:54.717079 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70547b27-ab43-4ade-a9be-7662802e73cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:54.754060 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-70547b27-ab43-4ade-a9be-7662802e73cd None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:18:54.755456 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-70547b27-ab43-4ade-a9be-7662802e73cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:18:54.855392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 11/23] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:18:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:18:54.892734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85765276-dfb6-4f0a-a2e7-b488ad739592 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:55.033762 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85765276-dfb6-4f0a-a2e7-b488ad739592 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:55.079150 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df697b0f-b8c6-437d-b04a-ec4c50f6c95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:55.079956 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df697b0f-b8c6-437d-b04a-ec4c50f6c95b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:55.080240 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df697b0f-b8c6-437d-b04a-ec4c50f6c95b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:55.080515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df697b0f-b8c6-437d-b04a-ec4c50f6c95b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:55.102533 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df697b0f-b8c6-437d-b04a-ec4c50f6c95b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:55.104027 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df697b0f-b8c6-437d-b04a-ec4c50f6c95b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:55.135690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-df697b0f-b8c6-437d-b04a-ec4c50f6c95b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:18:55.135690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:18:55.135690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:18:55.135690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:18:55.135690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:18:55.135690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:18:55.135690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:18:55.135690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:18:55.135690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:18:55.135690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:18:55.135690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:18:55.135690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:18:55.135690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:18:55.135690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:18:55.135690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:18:55.136528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:18:55.136528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:18:55.136528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:18:55.136528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:18:55.136528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:18:55.136528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:18:55.136528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:18:55.136528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:18:55.136528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:18:55.136528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:18:55.136528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:18:55.136528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:18:55.136528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:18:55.136528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:18:55.136528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:18:55.136528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:18:55.136528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:18:55.137365 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:18:55.137365 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:18:55.137365 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:18:55.137365 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:18:55.137365 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 13/24] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:18:55.143081 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70547b27-ab43-4ade-a9be-7662802e73cd None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:55.177351 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:55.206604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039fad45-ce28-4aa1-ae1a-8248930e5323 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:55.206911 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039fad45-ce28-4aa1-ae1a-8248930e5323 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:55.207166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039fad45-ce28-4aa1-ae1a-8248930e5323 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:55.208626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 14/25] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:18:55.219270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a9691-7fc2-4582-a4a4-352c23292e4b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:55.219555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a9691-7fc2-4582-a4a4-352c23292e4b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:55.219835 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a9691-7fc2-4582-a4a4-352c23292e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:55.249161 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1b6a9691-7fc2-4582-a4a4-352c23292e4b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:18:55.250862 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1b6a9691-7fc2-4582-a4a4-352c23292e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:18:55.281242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:55.337131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70547b27-ab43-4ade-a9be-7662802e73cd None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:55.413718 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438ea8b-b06d-4827-9da7-0b35421cd004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:55.414509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438ea8b-b06d-4827-9da7-0b35421cd004 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:55.414792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438ea8b-b06d-4827-9da7-0b35421cd004 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:55.415069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438ea8b-b06d-4827-9da7-0b35421cd004 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:55.415320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 15/26] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:18:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:18:55.415805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b438ea8b-b06d-4827-9da7-0b35421cd004 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:18:55.416111 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b438ea8b-b06d-4827-9da7-0b35421cd004 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:55.417547 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b438ea8b-b06d-4827-9da7-0b35421cd004 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:55.433860 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 12/27] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:18:55.439680 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a9691-7fc2-4582-a4a4-352c23292e4b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:55.456549 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438ea8b-b06d-4827-9da7-0b35421cd004 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:55.605430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:55.628009 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438ea8b-b06d-4827-9da7-0b35421cd004 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:55.677628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca2cf0-0d49-4159-b1c5-b7ffbeab2c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:55.679640 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca2cf0-0d49-4159-b1c5-b7ffbeab2c9b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:55.681695 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a9691-7fc2-4582-a4a4-352c23292e4b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:55.682182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca2cf0-0d49-4159-b1c5-b7ffbeab2c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:55.682663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca2cf0-0d49-4159-b1c5-b7ffbeab2c9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:55.706906 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca2cf0-0d49-4159-b1c5-b7ffbeab2c9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:18:55.715215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-22ca2cf0-0d49-4159-b1c5-b7ffbeab2c9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:18:55.715215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:18:55.715215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:18:55.715215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:18:55.715215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:18:55.715215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:18:55.715215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:18:55.715215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:18:55.715215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:18:55.715215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:18:55.715215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:18:55.715215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:18:55.715215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:18:55.715215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:18:55.715215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:18:55.715215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:18:55.716019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:18:55.716019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:18:55.716019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:18:55.716019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:18:55.716019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:18:55.716019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:18:55.716019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:18:55.716019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:18:55.716019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:18:55.716019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:18:55.716019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:18:55.716019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:18:55.716019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:18:55.716019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:18:55.716019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:18:55.716019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:18:55.717214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:18:55.717214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:18:55.717214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:18:55.717214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:18:55.717214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:18:55.717214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:18:55.717214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:18:55.717214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:18:55.717214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:18:55.717214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:18:55.717214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:18:55.717214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:18:55.730242 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 13/28] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:18:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:18:55.734303 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a34747-9b23-47d0-89bf-b784fe951769 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:55.735541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a34747-9b23-47d0-89bf-b784fe951769 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:55.736033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a34747-9b23-47d0-89bf-b784fe951769 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:55.740243 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 14/29] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:18:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:18:55.741272 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a34747-9b23-47d0-89bf-b784fe951769 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:55.766236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b833bb-9c97-4c8d-9fba-b850e3aacbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:55.766926 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b833bb-9c97-4c8d-9fba-b850e3aacbaf None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:55.767219 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b833bb-9c97-4c8d-9fba-b850e3aacbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:55.767506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b833bb-9c97-4c8d-9fba-b850e3aacbaf None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:55.768248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b833bb-9c97-4c8d-9fba-b850e3aacbaf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:55.769760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b833bb-9c97-4c8d-9fba-b850e3aacbaf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:55.872467 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a34747-9b23-47d0-89bf-b784fe951769 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:55.909646 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a56da-f269-48b5-8e3e-95d6918825c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:55.910662 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a56da-f269-48b5-8e3e-95d6918825c9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:55.911138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a56da-f269-48b5-8e3e-95d6918825c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:55.911602 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a56da-f269-48b5-8e3e-95d6918825c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:55.912654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a56da-f269-48b5-8e3e-95d6918825c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:18:55.913168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a56da-f269-48b5-8e3e-95d6918825c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:55.914683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a56da-f269-48b5-8e3e-95d6918825c9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:55.928363 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 15/30] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:18:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:18:55.930087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee00c7-1bda-4795-abe0-ec7091c35db7 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:55.930603 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee00c7-1bda-4795-abe0-ec7091c35db7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:55.931086 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee00c7-1bda-4795-abe0-ec7091c35db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:55.942971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:55.945566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:55.949923 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-5bee00c7-1bda-4795-abe0-ec7091c35db7 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:18:55.951325 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5bee00c7-1bda-4795-abe0-ec7091c35db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:18:56.025592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 16/31] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:18:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:18:56.027106 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65f175-59ab-4a1e-bde8-0e02c5037235 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:56.027524 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65f175-59ab-4a1e-bde8-0e02c5037235 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:56.027890 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65f175-59ab-4a1e-bde8-0e02c5037235 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:56.033631 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 17/32] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:18:56.034687 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65f175-59ab-4a1e-bde8-0e02c5037235 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:56.035128 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 16/33] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jun 7 08:18:55 2022] POST /identity/v3/projects => generated 301 bytes in 596 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:18:56.036793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78db94f-54a7-45dc-b5b8-933203b8b35f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:56.037132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78db94f-54a7-45dc-b5b8-933203b8b35f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:56.037415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78db94f-54a7-45dc-b5b8-933203b8b35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:56.055899 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b78db94f-54a7-45dc-b5b8-933203b8b35f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:18:56.057324 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b78db94f-54a7-45dc-b5b8-933203b8b35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:18:56.138355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 17/34] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:18:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:18:56.139219 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78db94f-54a7-45dc-b5b8-933203b8b35f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:56.184428 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65f175-59ab-4a1e-bde8-0e02c5037235 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:56.236296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b75e62-8e6b-4f2b-a147-58485b829a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:56.237273 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b75e62-8e6b-4f2b-a147-58485b829a7f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:56.237557 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b75e62-8e6b-4f2b-a147-58485b829a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:56.237832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b75e62-8e6b-4f2b-a147-58485b829a7f None admin] PATH_INFO: `/v3/users/4ce5a782a83f41fbb386c88e6d77ba55` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:56.239236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b75e62-8e6b-4f2b-a147-58485b829a7f None admin] RBAC: Authorizing `identity:get_user(user_id=4ce5a782a83f41fbb386c88e6d77ba55)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:56.241803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b75e62-8e6b-4f2b-a147-58485b829a7f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:56.244801 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 18/35] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:18:56 2022] GET /identity/v3/users/4ce5a782a83f41fbb386c88e6d77ba55 => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:18:56.245727 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b75e62-8e6b-4f2b-a147-58485b829a7f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:56.253580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:56.258131 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78db94f-54a7-45dc-b5b8-933203b8b35f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:56.294431 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:56.302187 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae374ae-a29c-4837-b657-0d191e40db48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:56.303014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae374ae-a29c-4837-b657-0d191e40db48 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:56.303446 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae374ae-a29c-4837-b657-0d191e40db48 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:56.303794 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae374ae-a29c-4837-b657-0d191e40db48 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:56.304637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae374ae-a29c-4837-b657-0d191e40db48 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:56.306403 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae374ae-a29c-4837-b657-0d191e40db48 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:56.370643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b75e62-8e6b-4f2b-a147-58485b829a7f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:56.404232 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 18/36] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Jun 7 08:18:56 2022] POST /identity/v3/roles => generated 219 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:18:56.404983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae374ae-a29c-4837-b657-0d191e40db48 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:56.417390 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bc08c-13e4-46aa-97af-1fbed10b4cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:56.418538 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bc08c-13e4-46aa-97af-1fbed10b4cb0 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:56.419022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bc08c-13e4-46aa-97af-1fbed10b4cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:56.419509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bc08c-13e4-46aa-97af-1fbed10b4cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:56.421080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bc08c-13e4-46aa-97af-1fbed10b4cb0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:56.423984 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bc08c-13e4-46aa-97af-1fbed10b4cb0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:56.537427 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae374ae-a29c-4837-b657-0d191e40db48 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:56.574169 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404773a8-7893-4f34-8f9c-7caade1441d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:56.574975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404773a8-7893-4f34-8f9c-7caade1441d1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:56.575266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404773a8-7893-4f34-8f9c-7caade1441d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:56.575545 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404773a8-7893-4f34-8f9c-7caade1441d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:56.576774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404773a8-7893-4f34-8f9c-7caade1441d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:56.578343 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404773a8-7893-4f34-8f9c-7caade1441d1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:56.580209 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 19/37] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:18:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:18:56.585234 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d452f-9d99-4229-aea7-e1f3bde28a7c None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:56.585234 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d452f-9d99-4229-aea7-e1f3bde28a7c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:56.585234 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d452f-9d99-4229-aea7-e1f3bde28a7c None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:56.585234 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 20/38] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:18:56.585234 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12658973-898b-4eed-bdc6-bbf4a5755179 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:56.585234 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12658973-898b-4eed-bdc6-bbf4a5755179 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:56.585646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12658973-898b-4eed-bdc6-bbf4a5755179 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:56.586015 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 21/39] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:18:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:18:56.586691 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12658973-898b-4eed-bdc6-bbf4a5755179 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:56.598091 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 19/40] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Jun 7 08:18:56 2022] POST /identity/v3/roles => generated 225 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:18:56.605194 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661dc29-e3e7-46ee-9989-798795d9af8a None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:56.605194 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661dc29-e3e7-46ee-9989-798795d9af8a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:56.605194 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661dc29-e3e7-46ee-9989-798795d9af8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:56.630224 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-4661dc29-e3e7-46ee-9989-798795d9af8a None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:18:56.632372 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4661dc29-e3e7-46ee-9989-798795d9af8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:18:56.672369 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:56.717587 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:56.730258 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 20/41] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:18:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:18:56.731593 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d827b-6f7b-4b04-8ac5-461f2ffc2dad None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:56.731885 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d827b-6f7b-4b04-8ac5-461f2ffc2dad None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:56.732170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d827b-6f7b-4b04-8ac5-461f2ffc2dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:56.739449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12658973-898b-4eed-bdc6-bbf4a5755179 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:56.754557 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-2c3d827b-6f7b-4b04-8ac5-461f2ffc2dad None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:18:56.755836 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2c3d827b-6f7b-4b04-8ac5-461f2ffc2dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:18:56.785895 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46891681-dcd9-426b-9abf-09ba0aaee655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:56.786790 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46891681-dcd9-426b-9abf-09ba0aaee655 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:56.787072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46891681-dcd9-426b-9abf-09ba0aaee655 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:56.787349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46891681-dcd9-426b-9abf-09ba0aaee655 None admin] PATH_INFO: `/v3/domains/default/users/4ce5a782a83f41fbb386c88e6d77ba55/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:56.789436 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46891681-dcd9-426b-9abf-09ba0aaee655 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4ce5a782a83f41fbb386c88e6d77ba55, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:56.791013 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46891681-dcd9-426b-9abf-09ba0aaee655 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:56.833565 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 21/42] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:18:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:18:56.834528 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d827b-6f7b-4b04-8ac5-461f2ffc2dad None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:56.883267 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 22/43] 10.222.0.17 () {58 vars in 1361 bytes} [Tue Jun 7 08:18:56 2022] PUT /identity/v3/domains/default/users/4ce5a782a83f41fbb386c88e6d77ba55/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:18:56.884097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46891681-dcd9-426b-9abf-09ba0aaee655 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:56.949837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d827b-6f7b-4b04-8ac5-461f2ffc2dad None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:56.977764 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:56.989989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e757d6-f2a8-457a-a646-e44bee5be216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:56.995869 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e757d6-f2a8-457a-a646-e44bee5be216 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:56.995869 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e757d6-f2a8-457a-a646-e44bee5be216 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:56.995869 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e757d6-f2a8-457a-a646-e44bee5be216 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:56.995869 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e757d6-f2a8-457a-a646-e44bee5be216 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:56.995869 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e757d6-f2a8-457a-a646-e44bee5be216 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:57.010761 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46891681-dcd9-426b-9abf-09ba0aaee655 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:57.029596 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:57.050857 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2500f22-5947-4d8d-bc9d-4d6dd27ce25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:57.051666 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2500f22-5947-4d8d-bc9d-4d6dd27ce25c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:57.051951 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2500f22-5947-4d8d-bc9d-4d6dd27ce25c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:57.052729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2500f22-5947-4d8d-bc9d-4d6dd27ce25c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:57.053645 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2500f22-5947-4d8d-bc9d-4d6dd27ce25c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:57.055271 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2500f22-5947-4d8d-bc9d-4d6dd27ce25c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:57.056974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 23/44] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:18:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:18:57.063344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2500f22-5947-4d8d-bc9d-4d6dd27ce25c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:57.082440 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 22/45] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Jun 7 08:18:56 2022] POST /identity/v3/roles => generated 223 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:18:57.293541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2500f22-5947-4d8d-bc9d-4d6dd27ce25c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:57.293541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0743f4-3896-4c8f-bc0a-b260e7512358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:57.293541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0743f4-3896-4c8f-bc0a-b260e7512358 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:57.293541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0743f4-3896-4c8f-bc0a-b260e7512358 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:57.293985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0743f4-3896-4c8f-bc0a-b260e7512358 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:57.293985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0743f4-3896-4c8f-bc0a-b260e7512358 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:57.293985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0743f4-3896-4c8f-bc0a-b260e7512358 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:57.370274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 24/46] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jun 7 08:18:57 2022] POST /identity/v3/projects => generated 312 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:18:59.120591 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ecd80-c462-49af-9ec9-5d68f13fccae None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:59.121158 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ecd80-c462-49af-9ec9-5d68f13fccae None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:59.121158 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ecd80-c462-49af-9ec9-5d68f13fccae None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:59.122537 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 23/47] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:18:59.127849 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585ec5c-1979-4b92-91be-4c3f14b293c8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:59.128258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585ec5c-1979-4b92-91be-4c3f14b293c8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:59.128411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585ec5c-1979-4b92-91be-4c3f14b293c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:59.144477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b585ec5c-1979-4b92-91be-4c3f14b293c8 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:18:59.145688 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b585ec5c-1979-4b92-91be-4c3f14b293c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:18:59.178117 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:59.214168 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 25/48] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:18:59.238008 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ecd80-c462-49af-9ec9-5d68f13fccae None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:59.340365 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ecd80-c462-49af-9ec9-5d68f13fccae None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:59.372704 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983f3c0-28d7-430d-b385-19966c9e7b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:59.373451 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983f3c0-28d7-430d-b385-19966c9e7b59 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:59.373737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983f3c0-28d7-430d-b385-19966c9e7b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:59.374016 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983f3c0-28d7-430d-b385-19966c9e7b59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:59.383755 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983f3c0-28d7-430d-b385-19966c9e7b59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:18:59.391769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-4983f3c0-28d7-430d-b385-19966c9e7b59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:18:59.391769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:18:59.391769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:18:59.391769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:18:59.391769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:18:59.391769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:18:59.391769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:18:59.391769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:18:59.391769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:18:59.391769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:18:59.391769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:18:59.391769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:18:59.391769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:18:59.391769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:18:59.391769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:18:59.391769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:18:59.392595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:18:59.392595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:18:59.392595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:18:59.392595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:18:59.392595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:18:59.392595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:18:59.392595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:18:59.392595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:18:59.392595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:18:59.392595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:18:59.392595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:18:59.392595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:18:59.392595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:18:59.392595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:18:59.392595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:18:59.392595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:18:59.393407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:18:59.393407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:18:59.393407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:18:59.393407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:18:59.393407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:18:59.393407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:18:59.393407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:18:59.393407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:18:59.393407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:18:59.393407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:18:59.393407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:18:59.393407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:18:59.393407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 24/49] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:18:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:18:59.398706 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585ec5c-1979-4b92-91be-4c3f14b293c8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:59.493863 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585ec5c-1979-4b92-91be-4c3f14b293c8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:59.526261 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66ebf3-8475-4a99-9955-8c7fb418fd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:59.526926 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66ebf3-8475-4a99-9955-8c7fb418fd03 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:59.527213 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66ebf3-8475-4a99-9955-8c7fb418fd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:59.527490 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66ebf3-8475-4a99-9955-8c7fb418fd03 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:59.528236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66ebf3-8475-4a99-9955-8c7fb418fd03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:18:59.528544 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66ebf3-8475-4a99-9955-8c7fb418fd03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:59.529965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66ebf3-8475-4a99-9955-8c7fb418fd03 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:59.538445 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 26/50] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:18:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:18:59.543918 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983f3c0-28d7-430d-b385-19966c9e7b59 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:59.588543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:59.609045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d2651-f6be-4dc8-9703-b1e3b7190f85 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:59.609340 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d2651-f6be-4dc8-9703-b1e3b7190f85 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:59.609624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d2651-f6be-4dc8-9703-b1e3b7190f85 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:59.611252 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 27/51] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:18:59.616070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3a419-99e8-4b08-9a30-e27f9e368d6d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:59.617092 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3a419-99e8-4b08-9a30-e27f9e368d6d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:59.617092 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3a419-99e8-4b08-9a30-e27f9e368d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:59.633055 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-aed3a419-99e8-4b08-9a30-e27f9e368d6d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:18:59.634191 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-aed3a419-99e8-4b08-9a30-e27f9e368d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:18:59.645787 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983f3c0-28d7-430d-b385-19966c9e7b59 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:59.658314 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:59.690511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c72963-9b38-4b71-bd10-850bdc759ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:59.691399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c72963-9b38-4b71-bd10-850bdc759ecf None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:59.691691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c72963-9b38-4b71-bd10-850bdc759ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:59.691971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c72963-9b38-4b71-bd10-850bdc759ecf None admin] PATH_INFO: `/v3/users/4ce5a782a83f41fbb386c88e6d77ba55` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:59.693210 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c72963-9b38-4b71-bd10-850bdc759ecf None admin] RBAC: Authorizing `identity:get_user(user_id=4ce5a782a83f41fbb386c88e6d77ba55)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:59.694688 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c72963-9b38-4b71-bd10-850bdc759ecf None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:59.696312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 25/52] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:18:59 2022] GET /identity/v3/users/4ce5a782a83f41fbb386c88e6d77ba55 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:18:59.701529 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 28/53] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:18:59.701811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c72963-9b38-4b71-bd10-850bdc759ecf None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:59.725224 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3a419-99e8-4b08-9a30-e27f9e368d6d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:59.799512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c72963-9b38-4b71-bd10-850bdc759ecf None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:59.812405 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:59.821208 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3a419-99e8-4b08-9a30-e27f9e368d6d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:59.832989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b214f6f-74f6-468e-9299-bd04da10d12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:59.833812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b214f6f-74f6-468e-9299-bd04da10d12a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:59.834102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b214f6f-74f6-468e-9299-bd04da10d12a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:59.834387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b214f6f-74f6-468e-9299-bd04da10d12a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:59.835552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b214f6f-74f6-468e-9299-bd04da10d12a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:59.837062 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b214f6f-74f6-468e-9299-bd04da10d12a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:59.838645 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 26/54] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:18:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:18:59.844317 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b214f6f-74f6-468e-9299-bd04da10d12a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:59.854478 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baeaef5-bf02-4f5d-b4b4-b342f9115f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:59.855274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baeaef5-bf02-4f5d-b4b4-b342f9115f65 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:59.855564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baeaef5-bf02-4f5d-b4b4-b342f9115f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:59.855843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baeaef5-bf02-4f5d-b4b4-b342f9115f65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:59.857018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baeaef5-bf02-4f5d-b4b4-b342f9115f65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:59.858547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baeaef5-bf02-4f5d-b4b4-b342f9115f65 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:59.860050 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 29/55] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:18:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:18:59.865449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baeaef5-bf02-4f5d-b4b4-b342f9115f65 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:18:59.892719 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:18:59.941539 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b214f6f-74f6-468e-9299-bd04da10d12a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:59.961190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baeaef5-bf02-4f5d-b4b4-b342f9115f65 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:18:59.973847 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386d8c1-f3be-43e4-92cb-cb6e7bd5bc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:59.974494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386d8c1-f3be-43e4-92cb-cb6e7bd5bc25 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:59.974777 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386d8c1-f3be-43e4-92cb-cb6e7bd5bc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:59.975048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386d8c1-f3be-43e4-92cb-cb6e7bd5bc25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:59.975872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386d8c1-f3be-43e4-92cb-cb6e7bd5bc25 None admin] RBAC: Adding query filter params (role.id=8c63c00418a346b29f7db3f87a404e70, scope.domain.id=default, user.id=4ce5a782a83f41fbb386c88e6d77ba55) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:18:59.976182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386d8c1-f3be-43e4-92cb-cb6e7bd5bc25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:59.977645 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386d8c1-f3be-43e4-92cb-cb6e7bd5bc25 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:18:59.984744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 27/56] 10.222.0.17 () {56 vars in 1322 bytes} [Tue Jun 7 08:18:59 2022] GET /identity/v3/role_assignments?user.id=4ce5a782a83f41fbb386c88e6d77ba55&scope.domain.id=default&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 530 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:18:59.993412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee261a0-c7c0-40f4-ac4d-94ea79aa21f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:18:59.994096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee261a0-c7c0-40f4-ac4d-94ea79aa21f0 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:18:59.994382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee261a0-c7c0-40f4-ac4d-94ea79aa21f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:18:59.994657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee261a0-c7c0-40f4-ac4d-94ea79aa21f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:18:59.995380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee261a0-c7c0-40f4-ac4d-94ea79aa21f0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:18:59.996876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee261a0-c7c0-40f4-ac4d-94ea79aa21f0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:00.062355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 30/57] 10.222.0.17 () {60 vars in 1140 bytes} [Tue Jun 7 08:18:59 2022] POST /identity/v3/projects => generated 298 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:19:02.303182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69691d-3ff1-4034-91bc-18fcf765b78f None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:02.303671 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69691d-3ff1-4034-91bc-18fcf765b78f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:02.303752 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69691d-3ff1-4034-91bc-18fcf765b78f None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:02.305197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 28/58] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:02.310395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcfb91-cbc1-48fb-b445-e219380d66ff None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:02.310821 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcfb91-cbc1-48fb-b445-e219380d66ff None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:02.310960 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcfb91-cbc1-48fb-b445-e219380d66ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:02.327376 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-36bcfb91-cbc1-48fb-b445-e219380d66ff None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:02.328598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-36bcfb91-cbc1-48fb-b445-e219380d66ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:02.393090 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 31/59] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:02.417967 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb69691d-3ff1-4034-91bc-18fcf765b78f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:02.463714 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:02.516498 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb69691d-3ff1-4034-91bc-18fcf765b78f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:02.548565 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497d1a6-a40e-4881-8c4d-08674deb35c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:02.549363 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497d1a6-a40e-4881-8c4d-08674deb35c4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:02.549647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497d1a6-a40e-4881-8c4d-08674deb35c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:02.550177 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497d1a6-a40e-4881-8c4d-08674deb35c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:02.551330 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497d1a6-a40e-4881-8c4d-08674deb35c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:02.552785 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497d1a6-a40e-4881-8c4d-08674deb35c4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:02.554384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 29/60] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:19:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:02.559717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcfb91-cbc1-48fb-b445-e219380d66ff None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:02.570973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:02.654331 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcfb91-cbc1-48fb-b445-e219380d66ff None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:02.686826 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c72aed-2f23-43c6-bbb3-cf505ff043ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:02.687481 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c72aed-2f23-43c6-bbb3-cf505ff043ec None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:02.687767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c72aed-2f23-43c6-bbb3-cf505ff043ec None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:02.688047 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c72aed-2f23-43c6-bbb3-cf505ff043ec None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:02.688744 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c72aed-2f23-43c6-bbb3-cf505ff043ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:02.690226 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c72aed-2f23-43c6-bbb3-cf505ff043ec None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:02.779622 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 32/61] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jun 7 08:19:02 2022] POST /identity/v3/users => generated 275 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:19:06.204167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9788b5e-1262-415c-bb70-90d195382e70 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:06.204686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9788b5e-1262-415c-bb70-90d195382e70 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:06.204760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9788b5e-1262-415c-bb70-90d195382e70 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:06.206385 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 30/62] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:06.212687 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfde29-022d-469d-b0e8-9499fb7f9a21 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:06.213189 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfde29-022d-469d-b0e8-9499fb7f9a21 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:06.220390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfde29-022d-469d-b0e8-9499fb7f9a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:06.233381 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10726d6b-4e29-45d8-bba7-8dd32e960df7 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:06.234230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10726d6b-4e29-45d8-bba7-8dd32e960df7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:06.234488 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10726d6b-4e29-45d8-bba7-8dd32e960df7 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:06.236048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 31/63] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:06.241838 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b2134-b6e8-4515-b89c-40e9487e0cf0 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:06.242135 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b2134-b6e8-4515-b89c-40e9487e0cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:06.242421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b2134-b6e8-4515-b89c-40e9487e0cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:06.242798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-cebfde29-022d-469d-b0e8-9499fb7f9a21 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:06.244061 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cebfde29-022d-469d-b0e8-9499fb7f9a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:06.273688 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-bd1b2134-b6e8-4515-b89c-40e9487e0cf0 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:06.274901 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-bd1b2134-b6e8-4515-b89c-40e9487e0cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:06.369391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:06.382621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 33/64] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:06.410229 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfde29-022d-469d-b0e8-9499fb7f9a21 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:06.417369 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 32/65] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 07 08:19:06.418534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58141c72-7a37-4c80-8681-1cf57d6d36f3 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:06.418824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58141c72-7a37-4c80-8681-1cf57d6d36f3 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:06.419102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58141c72-7a37-4c80-8681-1cf57d6d36f3 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:06.420412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 33/66] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:06.427857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1419ad-32aa-4170-b387-ab7be768abb4 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:06.428161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1419ad-32aa-4170-b387-ab7be768abb4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:06.428418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1419ad-32aa-4170-b387-ab7be768abb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:06.455467 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-cf1419ad-32aa-4170-b387-ab7be768abb4 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:06.456569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cf1419ad-32aa-4170-b387-ab7be768abb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:06.497808 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:06.524189 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfde29-022d-469d-b0e8-9499fb7f9a21 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:06.528617 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 34/67] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:06.529729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1419ad-32aa-4170-b387-ab7be768abb4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:06.588049 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cc23e-3890-4397-870e-47b1cc91fab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:06.589329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cc23e-3890-4397-870e-47b1cc91fab6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:06.589647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cc23e-3890-4397-870e-47b1cc91fab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:06.590172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cc23e-3890-4397-870e-47b1cc91fab6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:06.606847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50cc23e-3890-4397-870e-47b1cc91fab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:06.619662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e50cc23e-3890-4397-870e-47b1cc91fab6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:06.619662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:06.619662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:06.619662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:06.619662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:06.619662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:06.619662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:06.619662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:06.619662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:06.619662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:06.619662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:06.619662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:06.619662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:06.619662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:06.619662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:06.619662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:06.620507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:06.620507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:06.620507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:06.620507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:06.620507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:06.620507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:06.620507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:06.620507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:06.620507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:06.620507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:06.620507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:06.620507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:06.620507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:06.620507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:06.620507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:06.620507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:06.621291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:06.621291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:06.621291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:06.621291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:06.621291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:06.621291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:06.621291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:06.621291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:06.621291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:06.621291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:06.621291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:06.621291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:06.621291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 34/68] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:19:06.621918 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cc23e-3890-4397-870e-47b1cc91fab6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:06.638646 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1419ad-32aa-4170-b387-ab7be768abb4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:06.676868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8dccd5-871d-40aa-a4c8-3a313692db95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:06.678033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dccd5-871d-40aa-a4c8-3a313692db95 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:06.678506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dccd5-871d-40aa-a4c8-3a313692db95 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:06.678969 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dccd5-871d-40aa-a4c8-3a313692db95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:06.692389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8dccd5-871d-40aa-a4c8-3a313692db95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:06.695460 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:06.704096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-af8dccd5-871d-40aa-a4c8-3a313692db95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:19:06.704096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:06.704096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:06.704096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:06.704096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:06.704096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:06.704096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:06.704096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:06.704096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:06.704096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:06.704096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:06.704096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:06.704096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:06.704096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:06.704096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:06.704096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:06.707482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:06.707482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:06.707482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:06.707482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:06.707482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:06.707482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:06.707482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:06.707482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:06.707482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:06.707482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:06.707482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:06.707482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:06.707482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:06.707482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:06.707482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:06.707482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:06.708281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:06.708281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:06.708281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:06.708281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:06.708281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:06.708281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:06.708281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:06.708281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:06.708281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:06.708281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:06.708281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:19:06.708281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:06.708281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 35/69] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:19:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:06.708281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8dccd5-871d-40aa-a4c8-3a313692db95 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:06.746480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:06.755481 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cc23e-3890-4397-870e-47b1cc91fab6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:06.788931 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee754c-5bf0-462a-9ddd-30318e2bd6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:06.789768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee754c-5bf0-462a-9ddd-30318e2bd6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:06.790055 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee754c-5bf0-462a-9ddd-30318e2bd6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:06.790339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee754c-5bf0-462a-9ddd-30318e2bd6f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:06.801176 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee754c-5bf0-462a-9ddd-30318e2bd6f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:06.809206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9aee754c-5bf0-462a-9ddd-30318e2bd6f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:06.809206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:06.809206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:06.809206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:06.809206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:06.809206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:06.809206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:06.809206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:06.809206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:06.809206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:06.809206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:06.809206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:06.809206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:06.809206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:06.809206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:06.809206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:06.810245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:06.810245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:06.810245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:06.810245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:06.810245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:06.810245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:06.810245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:06.810245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:06.810245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:06.810245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:06.810245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:06.810245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:06.810245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:06.810245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:06.810245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:06.810245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:06.811759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:06.811759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:06.811759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:06.811759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:06.811759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:06.811759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:06.811759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:06.811759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:06.811759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:06.811759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:06.811759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:06.811759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:06.811759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 35/70] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:19:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:06.813810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee754c-5bf0-462a-9ddd-30318e2bd6f3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:06.817614 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8dccd5-871d-40aa-a4c8-3a313692db95 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:06.849927 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04276cf6-64ab-4437-9166-bec345e82b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:06.850586 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04276cf6-64ab-4437-9166-bec345e82b0f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:06.850872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04276cf6-64ab-4437-9166-bec345e82b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:06.851215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04276cf6-64ab-4437-9166-bec345e82b0f None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:06.851972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04276cf6-64ab-4437-9166-bec345e82b0f None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:06.852275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04276cf6-64ab-4437-9166-bec345e82b0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:06.853699 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04276cf6-64ab-4437-9166-bec345e82b0f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:06.862379 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 36/71] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:06.863462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110e1be-b94d-4418-9cc9-f053ba04129e None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:06.863755 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110e1be-b94d-4418-9cc9-f053ba04129e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:06.864032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110e1be-b94d-4418-9cc9-f053ba04129e None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:06.865447 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 37/72] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:06.866352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490301d9-4e2d-415b-9fea-11f5a94674ac None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:06.866639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490301d9-4e2d-415b-9fea-11f5a94674ac None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:06.866924 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490301d9-4e2d-415b-9fea-11f5a94674ac None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:06.868133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 38/73] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:06.868829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490301d9-4e2d-415b-9fea-11f5a94674ac None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:06.918819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee754c-5bf0-462a-9ddd-30318e2bd6f3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:06.934351 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:06.951969 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b3f2c-94cb-416c-b639-88dbbc16c7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:06.953239 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b3f2c-94cb-416c-b639-88dbbc16c7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:06.953529 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b3f2c-94cb-416c-b639-88dbbc16c7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:06.953812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b3f2c-94cb-416c-b639-88dbbc16c7f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:06.954557 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b3f2c-94cb-416c-b639-88dbbc16c7f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:06.954868 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b3f2c-94cb-416c-b639-88dbbc16c7f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:06.956400 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b3f2c-94cb-416c-b639-88dbbc16c7f4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:06.961732 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:06.965152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 36/74] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:06.966118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b3f2c-94cb-416c-b639-88dbbc16c7f4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:06.969634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490301d9-4e2d-415b-9fea-11f5a94674ac None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:07.004420 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845983b3-09b3-494a-8706-52161143d98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:07.005233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845983b3-09b3-494a-8706-52161143d98a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:07.005515 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845983b3-09b3-494a-8706-52161143d98a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:07.005790 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845983b3-09b3-494a-8706-52161143d98a None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:07.006558 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845983b3-09b3-494a-8706-52161143d98a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:07.006853 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845983b3-09b3-494a-8706-52161143d98a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:07.008219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845983b3-09b3-494a-8706-52161143d98a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:07.016148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 39/75] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:19:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:07.017440 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c07f6c-3ce3-4685-aa5e-020cf02dc0b5 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:07.017727 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c07f6c-3ce3-4685-aa5e-020cf02dc0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:07.018007 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c07f6c-3ce3-4685-aa5e-020cf02dc0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:07.033643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-40c07f6c-3ce3-4685-aa5e-020cf02dc0b5 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:07.034817 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-40c07f6c-3ce3-4685-aa5e-020cf02dc0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:07.071723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b3f2c-94cb-416c-b639-88dbbc16c7f4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:07.099226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 40/76] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:07.102283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9fe44-bb5d-42f5-94a8-04980b359b35 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:07.102572 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9fe44-bb5d-42f5-94a8-04980b359b35 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:07.102849 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9fe44-bb5d-42f5-94a8-04980b359b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:07.104446 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af13d9-8352-4a26-9e9d-9d503e3f2969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:07.105341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af13d9-8352-4a26-9e9d-9d503e3f2969 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:07.105629 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af13d9-8352-4a26-9e9d-9d503e3f2969 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:07.105965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af13d9-8352-4a26-9e9d-9d503e3f2969 None admin] PATH_INFO: `/v3/users/73798cb88b144a3db46809da81f534cf` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:07.118941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7ef9fe44-bb5d-42f5-94a8-04980b359b35 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:07.120100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7ef9fe44-bb5d-42f5-94a8-04980b359b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:07.121129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af13d9-8352-4a26-9e9d-9d503e3f2969 None admin] RBAC: Authorizing `identity:get_user(user_id=73798cb88b144a3db46809da81f534cf)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:07.122719 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af13d9-8352-4a26-9e9d-9d503e3f2969 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:07.124904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 37/77] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:06 2022] GET /identity/v3/users/73798cb88b144a3db46809da81f534cf => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:07.124904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af13d9-8352-4a26-9e9d-9d503e3f2969 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:07.161882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:07.197021 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 41/78] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:07.197021 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9fe44-bb5d-42f5-94a8-04980b359b35 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:07.205932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:07.229368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af13d9-8352-4a26-9e9d-9d503e3f2969 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:07.263564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a04bde-670c-4550-87b8-27c1487fbe8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:07.264442 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a04bde-670c-4550-87b8-27c1487fbe8b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:07.264735 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a04bde-670c-4550-87b8-27c1487fbe8b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:07.265038 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a04bde-670c-4550-87b8-27c1487fbe8b None admin] PATH_INFO: `/v3/users/4ce5a782a83f41fbb386c88e6d77ba55` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:07.266247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a04bde-670c-4550-87b8-27c1487fbe8b None admin] RBAC: Authorizing `identity:get_user(user_id=4ce5a782a83f41fbb386c88e6d77ba55)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:07.267703 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a04bde-670c-4550-87b8-27c1487fbe8b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:07.269362 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 38/79] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:07 2022] GET /identity/v3/users/4ce5a782a83f41fbb386c88e6d77ba55 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:07.270125 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a04bde-670c-4550-87b8-27c1487fbe8b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:07.286250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9fe44-bb5d-42f5-94a8-04980b359b35 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:07.320642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea625951-8a49-4d0d-9ed5-0ecd1dc781f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:07.321527 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea625951-8a49-4d0d-9ed5-0ecd1dc781f1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:07.321810 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea625951-8a49-4d0d-9ed5-0ecd1dc781f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:07.322086 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea625951-8a49-4d0d-9ed5-0ecd1dc781f1 None admin] PATH_INFO: `/v3/users/73798cb88b144a3db46809da81f534cf` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:07.323261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea625951-8a49-4d0d-9ed5-0ecd1dc781f1 None admin] RBAC: Authorizing `identity:get_user(user_id=73798cb88b144a3db46809da81f534cf)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:07.324709 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea625951-8a49-4d0d-9ed5-0ecd1dc781f1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:07.326335 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 42/80] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:07 2022] GET /identity/v3/users/73798cb88b144a3db46809da81f534cf => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:07.327087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea625951-8a49-4d0d-9ed5-0ecd1dc781f1 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:07.372255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a04bde-670c-4550-87b8-27c1487fbe8b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:07.374583 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:07.407927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17242ea-bfed-40a7-98f8-0b5b731df05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:07.408724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17242ea-bfed-40a7-98f8-0b5b731df05a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:07.409033 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17242ea-bfed-40a7-98f8-0b5b731df05a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:07.409304 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17242ea-bfed-40a7-98f8-0b5b731df05a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:07.409808 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:07.410539 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17242ea-bfed-40a7-98f8-0b5b731df05a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:07.411926 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17242ea-bfed-40a7-98f8-0b5b731df05a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:07.413699 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 39/81] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:19:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:07.414303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17242ea-bfed-40a7-98f8-0b5b731df05a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:07.431786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea625951-8a49-4d0d-9ed5-0ecd1dc781f1 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:07.465543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde149b-3bd0-4709-8a78-909b7a4bce62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:07.466174 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde149b-3bd0-4709-8a78-909b7a4bce62 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:07.466457 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde149b-3bd0-4709-8a78-909b7a4bce62 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:07.466738 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde149b-3bd0-4709-8a78-909b7a4bce62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:07.510065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17242ea-bfed-40a7-98f8-0b5b731df05a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:07.534510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde149b-3bd0-4709-8a78-909b7a4bce62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:07.536005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde149b-3bd0-4709-8a78-909b7a4bce62 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:07.542746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522d855-65c7-480d-b380-a4750fb68daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:07.543537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522d855-65c7-480d-b380-a4750fb68daf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:07.543823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522d855-65c7-480d-b380-a4750fb68daf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:07.544101 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522d855-65c7-480d-b380-a4750fb68daf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:07.555085 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522d855-65c7-480d-b380-a4750fb68daf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:07.562698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e522d855-65c7-480d-b380-a4750fb68daf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:07.562698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:07.562698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:07.562698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:07.562698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:07.562698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:07.562698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:07.562698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:07.562698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:07.562698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:07.562698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:07.562698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:07.562698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:07.562698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:07.562698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:07.562698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:07.563508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:07.563508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:07.563508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:07.563508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:07.563508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:07.563508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:07.563508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:07.563508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:07.563508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:07.563508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:07.563508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:07.563508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:07.563508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:07.563508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:07.563508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:07.563508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:07.564258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:07.564258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:07.564258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:07.564258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:07.564258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:07.564258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:07.564258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:07.564258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:07.564258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:07.564258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:07.564258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:07.564258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:07.564258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 40/82] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:07.564882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522d855-65c7-480d-b380-a4750fb68daf None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:07.594810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:07.619842 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:07.635672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 43/83] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:07.636603 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde149b-3bd0-4709-8a78-909b7a4bce62 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:07.662643 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522d855-65c7-480d-b380-a4750fb68daf None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:07.696620 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1032815-d09d-4177-958a-822141c2d022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:07.697305 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1032815-d09d-4177-958a-822141c2d022 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:07.697590 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1032815-d09d-4177-958a-822141c2d022 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:07.697869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1032815-d09d-4177-958a-822141c2d022 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:07.735824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde149b-3bd0-4709-8a78-909b7a4bce62 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:07.763207 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1032815-d09d-4177-958a-822141c2d022 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:07.764620 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1032815-d09d-4177-958a-822141c2d022 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:07.768608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab059e-e3c8-46c8-8c3d-04e2cd83a49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:07.769282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab059e-e3c8-46c8-8c3d-04e2cd83a49f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:07.769570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab059e-e3c8-46c8-8c3d-04e2cd83a49f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:07.769848 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab059e-e3c8-46c8-8c3d-04e2cd83a49f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:07.793623 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:07.823229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:07.838885 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab059e-e3c8-46c8-8c3d-04e2cd83a49f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:07.840354 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab059e-e3c8-46c8-8c3d-04e2cd83a49f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:07.860836 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 41/84] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:07.865806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1032815-d09d-4177-958a-822141c2d022 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:07.937009 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 44/85] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:07.937980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab059e-e3c8-46c8-8c3d-04e2cd83a49f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:07.960980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1032815-d09d-4177-958a-822141c2d022 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:07.989356 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:07.994034 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698989e-c09c-4718-91f6-0d5f8e14f938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:07.994698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698989e-c09c-4718-91f6-0d5f8e14f938 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:07.994989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698989e-c09c-4718-91f6-0d5f8e14f938 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:07.995262 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698989e-c09c-4718-91f6-0d5f8e14f938 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:07.995969 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7698989e-c09c-4718-91f6-0d5f8e14f938 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:07.997499 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7698989e-c09c-4718-91f6-0d5f8e14f938 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:08.020681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:08.035414 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab059e-e3c8-46c8-8c3d-04e2cd83a49f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:08.067328 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a1619-18ff-4cbf-a2a5-5b23d77825d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:08.067983 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a1619-18ff-4cbf-a2a5-5b23d77825d4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:08.068266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a1619-18ff-4cbf-a2a5-5b23d77825d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:08.068544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a1619-18ff-4cbf-a2a5-5b23d77825d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:08.069314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a1619-18ff-4cbf-a2a5-5b23d77825d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:08.069616 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a1619-18ff-4cbf-a2a5-5b23d77825d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:08.071000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a1619-18ff-4cbf-a2a5-5b23d77825d4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:08.071267 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 42/86] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jun 7 08:19:07 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:19:08.072079 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698989e-c09c-4718-91f6-0d5f8e14f938 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:08.085502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 45/87] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:08.085502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a1619-18ff-4cbf-a2a5-5b23d77825d4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:08.196431 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698989e-c09c-4718-91f6-0d5f8e14f938 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:08.205791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a1619-18ff-4cbf-a2a5-5b23d77825d4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:08.234685 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6e740-400e-4a7e-a865-bccd9398c24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:08.235563 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6e740-400e-4a7e-a865-bccd9398c24d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:08.235848 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6e740-400e-4a7e-a865-bccd9398c24d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:08.236129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6e740-400e-4a7e-a865-bccd9398c24d None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:08.237464 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6e740-400e-4a7e-a865-bccd9398c24d None admin] RBAC: Authorizing `identity:get_project(project_id=fe9516226a724cf688d3d23e1392161d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:08.238903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6e740-400e-4a7e-a865-bccd9398c24d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:08.245813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 43/88] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:08 2022] GET /identity/v3/projects/fe9516226a724cf688d3d23e1392161d => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:08.245813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6e740-400e-4a7e-a865-bccd9398c24d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:08.253471 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7951338-e250-4397-a7cd-6eec94aff1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:08.254368 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7951338-e250-4397-a7cd-6eec94aff1bf None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:08.254687 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7951338-e250-4397-a7cd-6eec94aff1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:08.254967 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7951338-e250-4397-a7cd-6eec94aff1bf None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:08.256182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7951338-e250-4397-a7cd-6eec94aff1bf None admin] RBAC: Authorizing `identity:get_project(project_id=fe9516226a724cf688d3d23e1392161d)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:08.257682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7951338-e250-4397-a7cd-6eec94aff1bf None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:08.259586 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 46/89] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:08 2022] GET /identity/v3/projects/fe9516226a724cf688d3d23e1392161d => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:08.260189 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7951338-e250-4397-a7cd-6eec94aff1bf None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:08.289130 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:08.344129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:08.400133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7951338-e250-4397-a7cd-6eec94aff1bf None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:08.401356 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6e740-400e-4a7e-a865-bccd9398c24d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:08.446751 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90fa8b-07a0-4ad5-97a1-a308349ae782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:08.447637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90fa8b-07a0-4ad5-97a1-a308349ae782 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:08.447926 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90fa8b-07a0-4ad5-97a1-a308349ae782 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:08.448207 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90fa8b-07a0-4ad5-97a1-a308349ae782 None admin] PATH_INFO: `/v3/users/73798cb88b144a3db46809da81f534cf` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:08.449055 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac375b0a-9b4c-49c1-adc4-49e5b1a08067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:08.449492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90fa8b-07a0-4ad5-97a1-a308349ae782 None admin] RBAC: Authorizing `identity:get_user(user_id=73798cb88b144a3db46809da81f534cf)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:08.449926 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac375b0a-9b4c-49c1-adc4-49e5b1a08067 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:08.450216 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac375b0a-9b4c-49c1-adc4-49e5b1a08067 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:08.450499 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac375b0a-9b4c-49c1-adc4-49e5b1a08067 None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:08.450997 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90fa8b-07a0-4ad5-97a1-a308349ae782 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:08.451701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac375b0a-9b4c-49c1-adc4-49e5b1a08067 None admin] RBAC: Authorizing `identity:get_project(project_id=fe9516226a724cf688d3d23e1392161d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:08.452826 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 47/90] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:08 2022] GET /identity/v3/users/73798cb88b144a3db46809da81f534cf => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:08.453255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac375b0a-9b4c-49c1-adc4-49e5b1a08067 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:08.453530 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90fa8b-07a0-4ad5-97a1-a308349ae782 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:08.455193 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 44/91] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:08 2022] GET /identity/v3/projects/fe9516226a724cf688d3d23e1392161d => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:08.455970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac375b0a-9b4c-49c1-adc4-49e5b1a08067 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:08.576116 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:08.577635 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac375b0a-9b4c-49c1-adc4-49e5b1a08067 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:08.599799 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90fa8b-07a0-4ad5-97a1-a308349ae782 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:08.604497 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:08.619794 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f4d32-7a57-47cc-bde9-dbe3636f74e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:08.620600 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f4d32-7a57-47cc-bde9-dbe3636f74e2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:08.621048 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f4d32-7a57-47cc-bde9-dbe3636f74e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:08.621342 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f4d32-7a57-47cc-bde9-dbe3636f74e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:08.622444 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f4d32-7a57-47cc-bde9-dbe3636f74e2 None admin] RBAC: Adding query filter params (role.id=8c63c00418a346b29f7db3f87a404e70, scope.project.id=fe9516226a724cf688d3d23e1392161d, user.id=4ce5a782a83f41fbb386c88e6d77ba55) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:08.622769 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f4d32-7a57-47cc-bde9-dbe3636f74e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:08.625287 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f4d32-7a57-47cc-bde9-dbe3636f74e2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:08.633764 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 45/92] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:19:08 2022] GET /identity/v3/role_assignments?user.id=4ce5a782a83f41fbb386c88e6d77ba55&scope.project.id=fe9516226a724cf688d3d23e1392161d&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:08.634818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f4d32-7a57-47cc-bde9-dbe3636f74e2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:08.653763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef8f6b-1ae3-4e0b-92f8-b23de175be4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:08.654694 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef8f6b-1ae3-4e0b-92f8-b23de175be4e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:08.655056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef8f6b-1ae3-4e0b-92f8-b23de175be4e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:08.655447 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef8f6b-1ae3-4e0b-92f8-b23de175be4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:08.656368 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef8f6b-1ae3-4e0b-92f8-b23de175be4e None admin] RBAC: Adding query filter params (role.id=67111011e94c4e08928dd5e6b7a46af1, scope.project.id=fe9516226a724cf688d3d23e1392161d, user.id=73798cb88b144a3db46809da81f534cf) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:08.656754 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef8f6b-1ae3-4e0b-92f8-b23de175be4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:08.658333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef8f6b-1ae3-4e0b-92f8-b23de175be4e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:08.678414 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 48/93] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:19:08 2022] GET /identity/v3/role_assignments?user.id=73798cb88b144a3db46809da81f534cf&scope.project.id=fe9516226a724cf688d3d23e1392161d&role.id=67111011e94c4e08928dd5e6b7a46af1 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:08.679415 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef8f6b-1ae3-4e0b-92f8-b23de175be4e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:08.766560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f4d32-7a57-47cc-bde9-dbe3636f74e2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:08.812599 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97d2f7-6004-4f0b-b512-439eb47dc17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:08.813428 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97d2f7-6004-4f0b-b512-439eb47dc17c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:08.813719 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97d2f7-6004-4f0b-b512-439eb47dc17c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:08.813998 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97d2f7-6004-4f0b-b512-439eb47dc17c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:08.887633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:08.904634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97d2f7-6004-4f0b-b512-439eb47dc17c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:08.906746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97d2f7-6004-4f0b-b512-439eb47dc17c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:08.914389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef8f6b-1ae3-4e0b-92f8-b23de175be4e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:09.002334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570363e9-cef3-4634-a695-eed3e5917073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:09.003006 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570363e9-cef3-4634-a695-eed3e5917073 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:09.003298 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570363e9-cef3-4634-a695-eed3e5917073 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:09.003582 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570363e9-cef3-4634-a695-eed3e5917073 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:09.004434 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570363e9-cef3-4634-a695-eed3e5917073 None admin] RBAC: Adding query filter params (role.id=4552e69b96eb43f3ba6a9ca694d04ffb, scope.project.id=fe9516226a724cf688d3d23e1392161d, user.id=73798cb88b144a3db46809da81f534cf) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:09.004754 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570363e9-cef3-4634-a695-eed3e5917073 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:09.006252 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570363e9-cef3-4634-a695-eed3e5917073 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:09.026441 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 49/94] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:19:08 2022] GET /identity/v3/role_assignments?user.id=73798cb88b144a3db46809da81f534cf&scope.project.id=fe9516226a724cf688d3d23e1392161d&role.id=4552e69b96eb43f3ba6a9ca694d04ffb => generated 259 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:09.104520 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 46/95] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:09.110667 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570363e9-cef3-4634-a695-eed3e5917073 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:09.124175 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:09.246057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570363e9-cef3-4634-a695-eed3e5917073 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:09.288682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a36cc8-a168-4f15-8ea0-df4b05b3e874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:09.289647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a36cc8-a168-4f15-8ea0-df4b05b3e874 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:09.289937 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a36cc8-a168-4f15-8ea0-df4b05b3e874 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:09.290219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a36cc8-a168-4f15-8ea0-df4b05b3e874 None admin] PATH_INFO: `/v3/projects/ebd04f5343964f4c9fc70331b9208d72` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:09.293801 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a36cc8-a168-4f15-8ea0-df4b05b3e874 None admin] RBAC: Authorizing `identity:get_project(project_id=ebd04f5343964f4c9fc70331b9208d72)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:09.295493 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a36cc8-a168-4f15-8ea0-df4b05b3e874 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:09.297524 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 50/96] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:09 2022] GET /identity/v3/projects/ebd04f5343964f4c9fc70331b9208d72 => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:09.303405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97d2f7-6004-4f0b-b512-439eb47dc17c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:09.410986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:09.423770 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97d2f7-6004-4f0b-b512-439eb47dc17c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:09.470287 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b38ffe-fafa-4cea-87a9-adc03260280b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:09.470923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b38ffe-fafa-4cea-87a9-adc03260280b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:09.471280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b38ffe-fafa-4cea-87a9-adc03260280b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:09.471497 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b38ffe-fafa-4cea-87a9-adc03260280b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:09.472343 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b38ffe-fafa-4cea-87a9-adc03260280b None admin] RBAC: Adding query filter params (role.id=67111011e94c4e08928dd5e6b7a46af1, scope.project.id=ebd04f5343964f4c9fc70331b9208d72, user.id=73798cb88b144a3db46809da81f534cf) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:09.472664 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b38ffe-fafa-4cea-87a9-adc03260280b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:09.474152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b38ffe-fafa-4cea-87a9-adc03260280b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:09.486004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 47/97] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:19:09 2022] GET /identity/v3/role_assignments?user.id=73798cb88b144a3db46809da81f534cf&scope.project.id=ebd04f5343964f4c9fc70331b9208d72&role.id=67111011e94c4e08928dd5e6b7a46af1 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:10.974371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc3ece-b6c5-474f-a8de-cd04f91cfd19 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:10.974884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc3ece-b6c5-474f-a8de-cd04f91cfd19 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:10.974977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc3ece-b6c5-474f-a8de-cd04f91cfd19 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:10.976561 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 51/98] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:10.981504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f874bb-24bf-49e3-b288-d2ff8687e19f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:10.991894 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f874bb-24bf-49e3-b288-d2ff8687e19f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:10.991894 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f874bb-24bf-49e3-b288-d2ff8687e19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:11.004520 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-43f874bb-24bf-49e3-b288-d2ff8687e19f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:11.005776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-43f874bb-24bf-49e3-b288-d2ff8687e19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:11.097886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 48/99] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:11.131814 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc3ece-b6c5-474f-a8de-cd04f91cfd19 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:11.230244 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:11.258257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc3ece-b6c5-474f-a8de-cd04f91cfd19 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:11.261590 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c1a8d-6bb4-46dc-934b-6628cafecc4e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:11.261882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c1a8d-6bb4-46dc-934b-6628cafecc4e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:11.262169 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c1a8d-6bb4-46dc-934b-6628cafecc4e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:11.263728 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 49/100] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:19:11.272098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafff5d9-5683-4dd1-ba75-8db8b8c0f86b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:11.272387 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafff5d9-5683-4dd1-ba75-8db8b8c0f86b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:11.272672 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafff5d9-5683-4dd1-ba75-8db8b8c0f86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:11.301967 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-aafff5d9-5683-4dd1-ba75-8db8b8c0f86b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:11.303166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-aafff5d9-5683-4dd1-ba75-8db8b8c0f86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:11.357783 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6fcd3-7aa0-475a-8134-8c5e26871dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:11.359128 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6fcd3-7aa0-475a-8134-8c5e26871dbd None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:11.359456 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6fcd3-7aa0-475a-8134-8c5e26871dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:11.359738 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6fcd3-7aa0-475a-8134-8c5e26871dbd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:11.365718 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf6fcd3-7aa0-475a-8134-8c5e26871dbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:11.367211 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf6fcd3-7aa0-475a-8134-8c5e26871dbd None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:11.368852 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 52/101] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:19:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:11.374954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6fcd3-7aa0-475a-8134-8c5e26871dbd None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:11.459499 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 50/102] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:11.511166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafff5d9-5683-4dd1-ba75-8db8b8c0f86b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:11.531223 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:11.577712 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6fcd3-7aa0-475a-8134-8c5e26871dbd None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:11.627180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d29185-3627-405d-b42a-13b1130f3a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:11.628244 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d29185-3627-405d-b42a-13b1130f3a69 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:11.628742 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d29185-3627-405d-b42a-13b1130f3a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:11.629470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d29185-3627-405d-b42a-13b1130f3a69 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:11.630438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d29185-3627-405d-b42a-13b1130f3a69 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:11.632162 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d29185-3627-405d-b42a-13b1130f3a69 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:11.664269 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafff5d9-5683-4dd1-ba75-8db8b8c0f86b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:11.706078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f7998-a331-4e6e-8fe5-0f6d72fc4447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:11.707177 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f7998-a331-4e6e-8fe5-0f6d72fc4447 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:11.707601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f7998-a331-4e6e-8fe5-0f6d72fc4447 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:11.708015 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f7998-a331-4e6e-8fe5-0f6d72fc4447 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:11.712847 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:11.722066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f7998-a331-4e6e-8fe5-0f6d72fc4447 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:11.728725 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 53/103] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jun 7 08:19:11 2022] POST /identity/v3/projects => generated 302 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:19:11.730476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-319f7998-a331-4e6e-8fe5-0f6d72fc4447 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:19:11.730476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:11.730476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:11.730476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:11.730476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:11.730476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:11.730476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:11.730476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:11.730476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:11.730476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:11.730476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:11.730476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:11.730476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:11.730476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:11.730476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:11.730476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:11.731323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:11.731323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:11.731323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:11.731323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:11.731323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:11.731323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:11.731323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:11.731323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:11.731323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:11.731323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:11.731323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:11.731323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:11.731323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:11.731323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:11.731323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:11.731323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:11.737118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:11.737118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:11.737118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:11.737118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:11.737118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:11.737118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:11.737118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:11.737118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:11.737118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:11.737118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:11.737118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:19:11.737118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:11.737118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 51/104] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:19:11.737824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5c054-0f8f-41da-bbf9-fb6ae7c29fb8 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:11.737824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5c054-0f8f-41da-bbf9-fb6ae7c29fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:11.737824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5c054-0f8f-41da-bbf9-fb6ae7c29fb8 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:11.740082 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f7998-a331-4e6e-8fe5-0f6d72fc4447 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:11.741788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 54/105] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:11 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:11.748096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ce8a9-4891-47bf-8856-b4040dba860d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:11.749327 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ce8a9-4891-47bf-8856-b4040dba860d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:11.749327 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ce8a9-4891-47bf-8856-b4040dba860d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:11.789348 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-707ce8a9-4891-47bf-8856-b4040dba860d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:11.789348 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-707ce8a9-4891-47bf-8856-b4040dba860d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:11.896471 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:11.926819 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 55/106] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:11.964684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ce8a9-4891-47bf-8856-b4040dba860d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:11.972585 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f7998-a331-4e6e-8fe5-0f6d72fc4447 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:12.081065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d46d37-2eef-4cb3-858c-4b114f3fe273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:12.081662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d46d37-2eef-4cb3-858c-4b114f3fe273 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:12.081988 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d46d37-2eef-4cb3-858c-4b114f3fe273 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:12.082272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d46d37-2eef-4cb3-858c-4b114f3fe273 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:12.083028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d46d37-2eef-4cb3-858c-4b114f3fe273 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:12.083337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d46d37-2eef-4cb3-858c-4b114f3fe273 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:12.084788 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d46d37-2eef-4cb3-858c-4b114f3fe273 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:12.098089 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 52/107] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:12.099627 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ee0f1-c923-4e9a-9c82-5acc02951a0e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:12.100065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ee0f1-c923-4e9a-9c82-5acc02951a0e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:12.100490 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ee0f1-c923-4e9a-9c82-5acc02951a0e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:12.102173 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 53/108] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:12.103384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ce8a9-4891-47bf-8856-b4040dba860d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:12.107018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ee0f1-c923-4e9a-9c82-5acc02951a0e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:12.143272 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:12.158633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab712a-d5f2-47d6-a032-a59d937e6c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:12.159593 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab712a-d5f2-47d6-a032-a59d937e6c89 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:12.160068 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab712a-d5f2-47d6-a032-a59d937e6c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:12.160663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab712a-d5f2-47d6-a032-a59d937e6c89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:12.178932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab712a-d5f2-47d6-a032-a59d937e6c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:12.190029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-1eab712a-d5f2-47d6-a032-a59d937e6c89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:19:12.190029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:12.190029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:12.190029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:12.190029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:12.190029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:12.190029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:12.190029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:12.190029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:12.190029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:12.190029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:12.190029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:12.190029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:12.190029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:12.190029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:12.190029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:12.190887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:12.190887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:12.190887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:12.190887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:12.190887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:12.190887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:12.190887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:12.190887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:12.190887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:12.190887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:12.190887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:12.190887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:12.190887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:12.190887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:12.190887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:12.190887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:12.191615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:12.191615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:12.191615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:12.191615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:12.191615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:12.191615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:12.191615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:12.191615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:12.191615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:12.191615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:12.191615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:19:12.191615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:12.191615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 56/109] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:19:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:12.192234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e34e8-d402-4ea1-bbab-90ab35a29a71 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:12.192234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e34e8-d402-4ea1-bbab-90ab35a29a71 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:12.192461 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e34e8-d402-4ea1-bbab-90ab35a29a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:12.225025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-458e34e8-d402-4ea1-bbab-90ab35a29a71 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:12.226441 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-458e34e8-d402-4ea1-bbab-90ab35a29a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:12.231661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ee0f1-c923-4e9a-9c82-5acc02951a0e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:12.302762 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15f2a8-ca43-40b3-9f78-5dd83eec89d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:12.303651 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15f2a8-ca43-40b3-9f78-5dd83eec89d1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:12.303936 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15f2a8-ca43-40b3-9f78-5dd83eec89d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:12.304217 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15f2a8-ca43-40b3-9f78-5dd83eec89d1 None admin] PATH_INFO: `/v3/users/1bb96419cbb049ee866966140d0ef35a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:12.307811 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:12.329840 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc15f2a8-ca43-40b3-9f78-5dd83eec89d1 None admin] RBAC: Authorizing `identity:get_user(user_id=1bb96419cbb049ee866966140d0ef35a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:12.331390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc15f2a8-ca43-40b3-9f78-5dd83eec89d1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:12.333376 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 54/110] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:12 2022] GET /identity/v3/users/1bb96419cbb049ee866966140d0ef35a => generated 289 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:12.334321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15f2a8-ca43-40b3-9f78-5dd83eec89d1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:12.373602 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 57/111] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:12.376076 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e34e8-d402-4ea1-bbab-90ab35a29a71 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:12.502826 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e34e8-d402-4ea1-bbab-90ab35a29a71 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:12.511898 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15f2a8-ca43-40b3-9f78-5dd83eec89d1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:12.520576 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:12.566969 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b50241-95be-463f-a08d-d073c52e6f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:12.568738 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b50241-95be-463f-a08d-d073c52e6f7e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:12.569664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b50241-95be-463f-a08d-d073c52e6f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:12.570514 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b50241-95be-463f-a08d-d073c52e6f7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:12.573767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8aabf-21d0-4b34-89c8-0b0903a59a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:12.574477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8aabf-21d0-4b34-89c8-0b0903a59a2f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:12.574748 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8aabf-21d0-4b34-89c8-0b0903a59a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:12.575029 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8aabf-21d0-4b34-89c8-0b0903a59a2f None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:12.575782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8aabf-21d0-4b34-89c8-0b0903a59a2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:12.576273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8aabf-21d0-4b34-89c8-0b0903a59a2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:12.577877 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8aabf-21d0-4b34-89c8-0b0903a59a2f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:12.594219 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 55/112] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:12.595304 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8aabf-21d0-4b34-89c8-0b0903a59a2f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:12.701389 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:12.707430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b50241-95be-463f-a08d-d073c52e6f7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:12.708939 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b50241-95be-463f-a08d-d073c52e6f7e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:12.769063 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8aabf-21d0-4b34-89c8-0b0903a59a2f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:12.809797 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb99b21-213b-4c4f-a196-6e75a058fc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:12.811576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb99b21-213b-4c4f-a196-6e75a058fc7d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:12.811576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb99b21-213b-4c4f-a196-6e75a058fc7d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:12.811576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb99b21-213b-4c4f-a196-6e75a058fc7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:12.829985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb99b21-213b-4c4f-a196-6e75a058fc7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:12.839424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-ecb99b21-213b-4c4f-a196-6e75a058fc7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:12.839424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:12.839424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:12.839424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:12.839424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:12.839424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:12.839424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:12.839424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:12.839424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:12.839424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:12.839424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:12.839424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:12.839424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:12.839424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:12.839424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:12.839424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:12.840327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:12.840327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:12.840327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:12.840327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:12.840327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:12.840327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:12.840327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:12.840327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:12.840327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:12.840327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:12.840327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:12.840327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:12.840327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:12.840327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:12.840327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:12.840327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:12.841160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:12.841160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:12.841160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:12.841160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:12.841160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:12.841160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:12.841160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:12.841160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:12.841160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:12.841160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:12.841160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:12.841160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:12.841160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 56/113] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:12.841811 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb99b21-213b-4c4f-a196-6e75a058fc7d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:12.943463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 58/114] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:12.944489 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b50241-95be-463f-a08d-d073c52e6f7e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:12.949250 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb99b21-213b-4c4f-a196-6e75a058fc7d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:12.970897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:12.985080 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3ae50-aa86-4dcd-bfbb-1c43e9e75ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:12.991830 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:12.992121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3ae50-aa86-4dcd-bfbb-1c43e9e75ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:12.992121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3ae50-aa86-4dcd-bfbb-1c43e9e75ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:12.992121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3ae50-aa86-4dcd-bfbb-1c43e9e75ac0 None admin] PATH_INFO: `/v3/users/4ce5a782a83f41fbb386c88e6d77ba55` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:12.992121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3ae50-aa86-4dcd-bfbb-1c43e9e75ac0 None admin] RBAC: Authorizing `identity:get_user(user_id=4ce5a782a83f41fbb386c88e6d77ba55)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:12.992121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3ae50-aa86-4dcd-bfbb-1c43e9e75ac0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:12.992593 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 57/115] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:12 2022] GET /identity/v3/users/4ce5a782a83f41fbb386c88e6d77ba55 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:12.994232 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3ae50-aa86-4dcd-bfbb-1c43e9e75ac0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:13.102807 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b50241-95be-463f-a08d-d073c52e6f7e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:13.115365 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3ae50-aa86-4dcd-bfbb-1c43e9e75ac0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:13.136856 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0c432-1905-4f6d-8c79-dbd53f716d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:13.137547 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0c432-1905-4f6d-8c79-dbd53f716d33 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:13.137833 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0c432-1905-4f6d-8c79-dbd53f716d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:13.138116 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0c432-1905-4f6d-8c79-dbd53f716d33 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:13.138859 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0c432-1905-4f6d-8c79-dbd53f716d33 None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:13.139167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0c432-1905-4f6d-8c79-dbd53f716d33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:13.141087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0c432-1905-4f6d-8c79-dbd53f716d33 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:13.148977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 59/116] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:13.149964 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0c432-1905-4f6d-8c79-dbd53f716d33 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:13.157670 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb313a8-a7aa-49ca-bb0e-e1a3340e7485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:13.158844 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb313a8-a7aa-49ca-bb0e-e1a3340e7485 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:13.159353 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb313a8-a7aa-49ca-bb0e-e1a3340e7485 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:13.159840 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb313a8-a7aa-49ca-bb0e-e1a3340e7485 None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:13.161477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb313a8-a7aa-49ca-bb0e-e1a3340e7485 None admin] RBAC: Authorizing `identity:get_project(project_id=fe9516226a724cf688d3d23e1392161d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:13.163250 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb313a8-a7aa-49ca-bb0e-e1a3340e7485 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:13.165284 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 58/117] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:12 2022] GET /identity/v3/projects/fe9516226a724cf688d3d23e1392161d => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:13.166857 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b689d-4a74-44ed-ad92-ff3ba704f4e9 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:13.167197 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b689d-4a74-44ed-ad92-ff3ba704f4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:13.167483 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b689d-4a74-44ed-ad92-ff3ba704f4e9 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:13.168704 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 59/118] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:13.169978 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83857dae-f136-47ac-86bf-3453f37a0ce5 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:13.170429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83857dae-f136-47ac-86bf-3453f37a0ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:13.170718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83857dae-f136-47ac-86bf-3453f37a0ce5 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:13.176406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 60/119] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:13.177439 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83857dae-f136-47ac-86bf-3453f37a0ce5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:13.236720 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:13.253396 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:13.272787 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0c432-1905-4f6d-8c79-dbd53f716d33 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:13.290641 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83857dae-f136-47ac-86bf-3453f37a0ce5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:13.310215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7eea1-e8c5-452c-9bde-d8917dac07a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:13.311334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7eea1-e8c5-452c-9bde-d8917dac07a1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:13.311454 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7eea1-e8c5-452c-9bde-d8917dac07a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:13.312191 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7eea1-e8c5-452c-9bde-d8917dac07a1 None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:13.313984 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7eea1-e8c5-452c-9bde-d8917dac07a1 None admin] RBAC: Authorizing `identity:get_project(project_id=fe9516226a724cf688d3d23e1392161d)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:13.316743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7eea1-e8c5-452c-9bde-d8917dac07a1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:13.322998 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 60/120] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:13 2022] GET /identity/v3/projects/fe9516226a724cf688d3d23e1392161d => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:13.322998 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7eea1-e8c5-452c-9bde-d8917dac07a1 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:13.336049 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bef53-864f-470a-85d1-b9a985d611eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:13.336940 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bef53-864f-470a-85d1-b9a985d611eb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:13.337261 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bef53-864f-470a-85d1-b9a985d611eb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:13.337542 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bef53-864f-470a-85d1-b9a985d611eb None admin] PATH_INFO: `/v3/users/73798cb88b144a3db46809da81f534cf` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:13.338739 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0bef53-864f-470a-85d1-b9a985d611eb None admin] RBAC: Authorizing `identity:get_user(user_id=73798cb88b144a3db46809da81f534cf)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:13.340183 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0bef53-864f-470a-85d1-b9a985d611eb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:13.344317 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 61/121] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:13 2022] GET /identity/v3/users/73798cb88b144a3db46809da81f534cf => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:13.344317 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff61e8-60c5-40c3-b0ea-3d8a0d51bc4b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:13.344317 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff61e8-60c5-40c3-b0ea-3d8a0d51bc4b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:13.344317 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff61e8-60c5-40c3-b0ea-3d8a0d51bc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:13.360525 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-46ff61e8-60c5-40c3-b0ea-3d8a0d51bc4b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:13.361761 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-46ff61e8-60c5-40c3-b0ea-3d8a0d51bc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:13.431285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7eea1-e8c5-452c-9bde-d8917dac07a1 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:13.433054 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 62/122] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:13.434398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224d3d8-caba-4261-a72d-9f6771a89f24 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:13.434638 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224d3d8-caba-4261-a72d-9f6771a89f24 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:13.434925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224d3d8-caba-4261-a72d-9f6771a89f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:13.462751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5224d3d8-caba-4261-a72d-9f6771a89f24 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:13.464005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5224d3d8-caba-4261-a72d-9f6771a89f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:13.471062 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf7d30-f13b-4d7d-aa2d-e8e63ced64bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:13.471794 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf7d30-f13b-4d7d-aa2d-e8e63ced64bb None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:13.472151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf7d30-f13b-4d7d-aa2d-e8e63ced64bb None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:13.472501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf7d30-f13b-4d7d-aa2d-e8e63ced64bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:13.473547 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf7d30-f13b-4d7d-aa2d-e8e63ced64bb None admin] RBAC: Adding query filter params (role.id=5733a653e21248a7ab8ff7624b5f8ce9, scope.project.id=fe9516226a724cf688d3d23e1392161d, user.id=1bb96419cbb049ee866966140d0ef35a) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:13.473747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf7d30-f13b-4d7d-aa2d-e8e63ced64bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:13.475278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf7d30-f13b-4d7d-aa2d-e8e63ced64bb None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:13.480825 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 61/123] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:19:13 2022] GET /identity/v3/role_assignments?user.id=1bb96419cbb049ee866966140d0ef35a&scope.project.id=fe9516226a724cf688d3d23e1392161d&role.id=5733a653e21248a7ab8ff7624b5f8ce9 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:13.481555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf7d30-f13b-4d7d-aa2d-e8e63ced64bb None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:13.497970 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:13.532482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:13.568222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 63/124] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:13.569525 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224d3d8-caba-4261-a72d-9f6771a89f24 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:13.598860 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf7d30-f13b-4d7d-aa2d-e8e63ced64bb None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:13.639036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f4657-551c-4c84-bd2f-514425f35cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:13.640089 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f4657-551c-4c84-bd2f-514425f35cef None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:13.640375 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f4657-551c-4c84-bd2f-514425f35cef None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:13.640655 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f4657-551c-4c84-bd2f-514425f35cef None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d/users/4ce5a782a83f41fbb386c88e6d77ba55/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:13.643020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f4657-551c-4c84-bd2f-514425f35cef None admin] RBAC: Authorizing `identity:create_grant(project_id=fe9516226a724cf688d3d23e1392161d, user_id=4ce5a782a83f41fbb386c88e6d77ba55, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:13.644527 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f4657-551c-4c84-bd2f-514425f35cef None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:13.699944 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224d3d8-caba-4261-a72d-9f6771a89f24 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:13.713263 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 62/125] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:19:13 2022] PUT /identity/v3/projects/fe9516226a724cf688d3d23e1392161d/users/4ce5a782a83f41fbb386c88e6d77ba55/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:19:13.722286 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f4657-551c-4c84-bd2f-514425f35cef None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:13.739280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b9a76-a1ef-4c1b-b700-014ff689ca7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:13.740656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b9a76-a1ef-4c1b-b700-014ff689ca7d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:13.741907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b9a76-a1ef-4c1b-b700-014ff689ca7d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:13.742468 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b9a76-a1ef-4c1b-b700-014ff689ca7d None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:13.744658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b9a76-a1ef-4c1b-b700-014ff689ca7d None admin] RBAC: Authorizing `identity:get_project(project_id=fe9516226a724cf688d3d23e1392161d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:13.747088 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b9a76-a1ef-4c1b-b700-014ff689ca7d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:13.750140 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 64/126] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:13 2022] GET /identity/v3/projects/fe9516226a724cf688d3d23e1392161d => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:13.751569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b9a76-a1ef-4c1b-b700-014ff689ca7d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:13.811244 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:13.891273 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:13.928385 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f4657-551c-4c84-bd2f-514425f35cef None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:13.959005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b9a76-a1ef-4c1b-b700-014ff689ca7d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:13.967143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3db8fc-d9a7-4ab4-903e-6bd6115c6c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:13.968528 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3db8fc-d9a7-4ab4-903e-6bd6115c6c2f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:13.968817 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3db8fc-d9a7-4ab4-903e-6bd6115c6c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:13.969120 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3db8fc-d9a7-4ab4-903e-6bd6115c6c2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:13.979971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3db8fc-d9a7-4ab4-903e-6bd6115c6c2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:13.990384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-4a3db8fc-d9a7-4ab4-903e-6bd6115c6c2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:13.990384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:13.990384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:13.990384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:13.990384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:13.990384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:13.990384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:13.990384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:13.990384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:13.990384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:13.990384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:13.990384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:13.990384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:13.990384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:13.990384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:13.990384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:13.991249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:13.991249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:13.991249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:13.991249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:13.991249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:13.991249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:13.991249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:13.991249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:13.991249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:13.991249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:13.991249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:13.991249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:13.991249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:13.991249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:13.991249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:13.991249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:13.992011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:13.992011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:13.992011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:13.992011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:13.992011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:13.992011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:13.992011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:13.992011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:13.992011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:13.992011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:13.992011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:13.992011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:13.992011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 63/127] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:13.992625 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3db8fc-d9a7-4ab4-903e-6bd6115c6c2f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:14.066702 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4022c6c-d6b9-42da-82bb-2963cac60d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:14.067455 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4022c6c-d6b9-42da-82bb-2963cac60d05 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:14.067736 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4022c6c-d6b9-42da-82bb-2963cac60d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:14.068008 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4022c6c-d6b9-42da-82bb-2963cac60d05 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:14.090040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4022c6c-d6b9-42da-82bb-2963cac60d05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:14.106070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e4022c6c-d6b9-42da-82bb-2963cac60d05 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:14.106070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:14.106070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:14.106070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:14.106070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:14.106070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:14.106070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:14.106070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:14.106070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:14.106070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:14.106070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:14.106070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:14.106070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:14.106070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:14.106070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:14.106070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:14.107000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:14.107000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:14.107000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:14.107000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:14.107000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:14.107000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:14.107000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:14.107000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:14.107000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:14.107000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:14.107000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:14.107000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:14.107000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:14.107000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:14.107000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:14.107000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:14.107746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:14.107746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:14.107746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:14.107746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:14.107746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:14.107746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:14.107746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:14.107746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:14.107746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:14.107746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:14.107746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:14.107746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:14.109822 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3db8fc-d9a7-4ab4-903e-6bd6115c6c2f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:14.111127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 65/128] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:19:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:14.112344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4022c6c-d6b9-42da-82bb-2963cac60d05 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:14.145481 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:14.150684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534b2d5-7b5c-45af-a38f-237d3b44ba94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:14.152173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534b2d5-7b5c-45af-a38f-237d3b44ba94 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:14.152569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534b2d5-7b5c-45af-a38f-237d3b44ba94 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:14.152951 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534b2d5-7b5c-45af-a38f-237d3b44ba94 None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d/users/73798cb88b144a3db46809da81f534cf/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:14.156253 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534b2d5-7b5c-45af-a38f-237d3b44ba94 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe9516226a724cf688d3d23e1392161d, user_id=73798cb88b144a3db46809da81f534cf, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:14.157924 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534b2d5-7b5c-45af-a38f-237d3b44ba94 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:14.249237 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4022c6c-d6b9-42da-82bb-2963cac60d05 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:14.260648 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 64/129] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:19:13 2022] PUT /identity/v3/projects/fe9516226a724cf688d3d23e1392161d/users/73798cb88b144a3db46809da81f534cf/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:19:14.261532 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534b2d5-7b5c-45af-a38f-237d3b44ba94 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:14.268455 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:14.322550 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775a805-7278-451c-b16f-31e2af0a3ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:14.324297 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775a805-7278-451c-b16f-31e2af0a3ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:14.324590 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775a805-7278-451c-b16f-31e2af0a3ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:14.326099 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775a805-7278-451c-b16f-31e2af0a3ad0 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:14.326893 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775a805-7278-451c-b16f-31e2af0a3ad0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:14.327242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775a805-7278-451c-b16f-31e2af0a3ad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:14.329648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775a805-7278-451c-b16f-31e2af0a3ad0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:14.344462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 66/130] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:14.350136 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775a805-7278-451c-b16f-31e2af0a3ad0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:14.454297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534b2d5-7b5c-45af-a38f-237d3b44ba94 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:14.492978 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775a805-7278-451c-b16f-31e2af0a3ad0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:14.533512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef53cba-60e1-4342-aa03-59cf5ddb50b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:14.534210 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef53cba-60e1-4342-aa03-59cf5ddb50b1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:14.534556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef53cba-60e1-4342-aa03-59cf5ddb50b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:14.534848 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef53cba-60e1-4342-aa03-59cf5ddb50b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:14.535620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef53cba-60e1-4342-aa03-59cf5ddb50b1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:14.535935 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef53cba-60e1-4342-aa03-59cf5ddb50b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:14.537378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef53cba-60e1-4342-aa03-59cf5ddb50b1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:14.547408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 65/131] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:19:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:14.548849 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c2ee2-5eb5-4388-98d9-59cfa8e0a374 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:14.549481 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c2ee2-5eb5-4388-98d9-59cfa8e0a374 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:14.549929 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c2ee2-5eb5-4388-98d9-59cfa8e0a374 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:14.551892 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 66/132] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:14.557438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c2ee2-5eb5-4388-98d9-59cfa8e0a374 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:14.623106 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:14.625754 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa424df9-47c3-41f7-88ff-87d41c57c8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:14.626662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa424df9-47c3-41f7-88ff-87d41c57c8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:14.626946 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa424df9-47c3-41f7-88ff-87d41c57c8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:14.627225 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa424df9-47c3-41f7-88ff-87d41c57c8f4 None admin] PATH_INFO: `/v3/users/73798cb88b144a3db46809da81f534cf` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:14.633896 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa424df9-47c3-41f7-88ff-87d41c57c8f4 None admin] RBAC: Authorizing `identity:get_user(user_id=73798cb88b144a3db46809da81f534cf)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:14.635390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa424df9-47c3-41f7-88ff-87d41c57c8f4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:14.642094 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 67/133] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:14 2022] GET /identity/v3/users/73798cb88b144a3db46809da81f534cf => generated 275 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:14.642094 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acf0dc-a4c5-4589-b92d-9b1fbc14ae63 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:14.642094 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acf0dc-a4c5-4589-b92d-9b1fbc14ae63 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:14.642094 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acf0dc-a4c5-4589-b92d-9b1fbc14ae63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:14.659542 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c1acf0dc-a4c5-4589-b92d-9b1fbc14ae63 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:14.660757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c1acf0dc-a4c5-4589-b92d-9b1fbc14ae63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:14.689302 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c2ee2-5eb5-4388-98d9-59cfa8e0a374 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:14.693360 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:14.732347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028b372-36a1-4fe3-a213-4f86555cef7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:14.733810 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028b372-36a1-4fe3-a213-4f86555cef7d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:14.734274 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028b372-36a1-4fe3-a213-4f86555cef7d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:14.734719 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028b372-36a1-4fe3-a213-4f86555cef7d None admin] PATH_INFO: `/v3/users/73798cb88b144a3db46809da81f534cf` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:14.735107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 68/134] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:14.736103 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1acf0dc-a4c5-4589-b92d-9b1fbc14ae63 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:14.736462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028b372-36a1-4fe3-a213-4f86555cef7d None admin] RBAC: Authorizing `identity:get_user(user_id=73798cb88b144a3db46809da81f534cf)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:14.738205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028b372-36a1-4fe3-a213-4f86555cef7d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:14.740975 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 67/135] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:14 2022] GET /identity/v3/users/73798cb88b144a3db46809da81f534cf => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:14.751570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028b372-36a1-4fe3-a213-4f86555cef7d None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:14.876638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028b372-36a1-4fe3-a213-4f86555cef7d None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:14.902772 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:14.906321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1acf0dc-a4c5-4589-b92d-9b1fbc14ae63 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:14.916618 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78fcd4-7fba-47b6-847f-2d9dfd3a8595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:14.919229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78fcd4-7fba-47b6-847f-2d9dfd3a8595 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:14.919229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78fcd4-7fba-47b6-847f-2d9dfd3a8595 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:14.919229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78fcd4-7fba-47b6-847f-2d9dfd3a8595 None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:14.920293 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78fcd4-7fba-47b6-847f-2d9dfd3a8595 None admin] RBAC: Authorizing `identity:get_project(project_id=fe9516226a724cf688d3d23e1392161d)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:14.922143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78fcd4-7fba-47b6-847f-2d9dfd3a8595 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:14.924466 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 68/136] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:14 2022] GET /identity/v3/projects/fe9516226a724cf688d3d23e1392161d => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:14.929028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78fcd4-7fba-47b6-847f-2d9dfd3a8595 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:14.944413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cd3c4-7e06-4b95-a72b-844ab5b6bee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:14.945269 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cd3c4-7e06-4b95-a72b-844ab5b6bee4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:14.945655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cd3c4-7e06-4b95-a72b-844ab5b6bee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:14.946027 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cd3c4-7e06-4b95-a72b-844ab5b6bee4 None admin] PATH_INFO: `/v3/projects/ebd04f5343964f4c9fc70331b9208d72` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:14.947403 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690cd3c4-7e06-4b95-a72b-844ab5b6bee4 None admin] RBAC: Authorizing `identity:get_project(project_id=ebd04f5343964f4c9fc70331b9208d72)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:14.948959 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690cd3c4-7e06-4b95-a72b-844ab5b6bee4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:14.951097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 69/137] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:14 2022] GET /identity/v3/projects/ebd04f5343964f4c9fc70331b9208d72 => generated 312 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:14.951683 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cd3c4-7e06-4b95-a72b-844ab5b6bee4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:14.986207 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:15.052452 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78fcd4-7fba-47b6-847f-2d9dfd3a8595 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:15.083290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cd3c4-7e06-4b95-a72b-844ab5b6bee4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:15.091425 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea746e-5a3b-440c-a22d-3ddfea3566c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:15.092233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea746e-5a3b-440c-a22d-3ddfea3566c4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:15.092522 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea746e-5a3b-440c-a22d-3ddfea3566c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:15.092837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea746e-5a3b-440c-a22d-3ddfea3566c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:15.094061 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea746e-5a3b-440c-a22d-3ddfea3566c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:15.095529 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea746e-5a3b-440c-a22d-3ddfea3566c4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:15.097479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 69/138] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:19:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:15.098247 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea746e-5a3b-440c-a22d-3ddfea3566c4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:15.119538 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d90068-118e-4586-9a2c-cfaed3c51483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:15.120602 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d90068-118e-4586-9a2c-cfaed3c51483 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:15.120990 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d90068-118e-4586-9a2c-cfaed3c51483 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:15.121402 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d90068-118e-4586-9a2c-cfaed3c51483 None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d/users/73798cb88b144a3db46809da81f534cf/roles/4552e69b96eb43f3ba6a9ca694d04ffb` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:15.123821 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d90068-118e-4586-9a2c-cfaed3c51483 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe9516226a724cf688d3d23e1392161d, user_id=73798cb88b144a3db46809da81f534cf, role_id=4552e69b96eb43f3ba6a9ca694d04ffb)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:15.126013 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d90068-118e-4586-9a2c-cfaed3c51483 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:15.165300 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:15.215595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 70/139] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:19:14 2022] PUT /identity/v3/projects/fe9516226a724cf688d3d23e1392161d/users/73798cb88b144a3db46809da81f534cf/roles/4552e69b96eb43f3ba6a9ca694d04ffb => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:19:15.216561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d90068-118e-4586-9a2c-cfaed3c51483 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:15.225382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea746e-5a3b-440c-a22d-3ddfea3566c4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:15.289842 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3784fc-cbd2-4e02-a9fe-481e2f7084fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:15.290770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3784fc-cbd2-4e02-a9fe-481e2f7084fa None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:15.291060 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3784fc-cbd2-4e02-a9fe-481e2f7084fa None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:15.291341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3784fc-cbd2-4e02-a9fe-481e2f7084fa None admin] PATH_INFO: `/v3/projects/ebd04f5343964f4c9fc70331b9208d72/users/73798cb88b144a3db46809da81f534cf/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:15.293588 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3784fc-cbd2-4e02-a9fe-481e2f7084fa None admin] RBAC: Authorizing `identity:create_grant(project_id=ebd04f5343964f4c9fc70331b9208d72, user_id=73798cb88b144a3db46809da81f534cf, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:15.295071 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3784fc-cbd2-4e02-a9fe-481e2f7084fa None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:15.361748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 70/140] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:19:15 2022] PUT /identity/v3/projects/ebd04f5343964f4c9fc70331b9208d72/users/73798cb88b144a3db46809da81f534cf/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:19:15.424550 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:15.432473 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d90068-118e-4586-9a2c-cfaed3c51483 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:15.481933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78117571-c41e-4c13-a75c-ce50736425b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:15.485851 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78117571-c41e-4c13-a75c-ce50736425b1 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:15.486148 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78117571-c41e-4c13-a75c-ce50736425b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:15.486438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78117571-c41e-4c13-a75c-ce50736425b1 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:15.487187 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78117571-c41e-4c13-a75c-ce50736425b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:15.488685 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78117571-c41e-4c13-a75c-ce50736425b1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:15.586516 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 71/141] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jun 7 08:19:15 2022] POST /identity/v3/users => generated 283 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:19:16.470687 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33c77c-43fb-42db-bf12-35a591f2d7dc None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:16.471162 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33c77c-43fb-42db-bf12-35a591f2d7dc None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:16.471162 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33c77c-43fb-42db-bf12-35a591f2d7dc None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:16.472846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 71/142] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:16.478158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c741199-4aed-4094-99c3-3452bba27521 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:16.478575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c741199-4aed-4094-99c3-3452bba27521 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:16.478761 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c741199-4aed-4094-99c3-3452bba27521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:16.502249 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9c741199-4aed-4094-99c3-3452bba27521 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:16.503464 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9c741199-4aed-4094-99c3-3452bba27521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:16.585665 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 72/143] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:16.598217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b144e7-59fa-4bf2-aa90-a07af4dda329 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:16.598509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b144e7-59fa-4bf2-aa90-a07af4dda329 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:16.598801 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b144e7-59fa-4bf2-aa90-a07af4dda329 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:16.600854 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 72/144] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:16.606221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066bd8b-d7d2-4b40-b30a-3c1ca1a721c7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:16.606515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066bd8b-d7d2-4b40-b30a-3c1ca1a721c7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:16.606801 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066bd8b-d7d2-4b40-b30a-3c1ca1a721c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:16.612127 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b144e7-59fa-4bf2-aa90-a07af4dda329 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:16.628436 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7066bd8b-d7d2-4b40-b30a-3c1ca1a721c7 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:16.629693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7066bd8b-d7d2-4b40-b30a-3c1ca1a721c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:16.704109 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 73/145] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:16.728645 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066bd8b-d7d2-4b40-b30a-3c1ca1a721c7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:16.741474 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b144e7-59fa-4bf2-aa90-a07af4dda329 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:16.749357 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:16.760305 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:16.779186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02767a38-ef75-4ee2-8085-aff96e8f4446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:16.779999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02767a38-ef75-4ee2-8085-aff96e8f4446 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:16.780842 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02767a38-ef75-4ee2-8085-aff96e8f4446 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:16.781165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02767a38-ef75-4ee2-8085-aff96e8f4446 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:16.805621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02767a38-ef75-4ee2-8085-aff96e8f4446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:16.813775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-02767a38-ef75-4ee2-8085-aff96e8f4446 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:19:16.813775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:16.813775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:16.813775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:16.813775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:16.813775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:16.813775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:16.813775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:16.813775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:16.813775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:16.813775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:16.813775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:16.813775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:16.813775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:16.813775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:16.813775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:16.814608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:16.814608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:16.814608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:16.814608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:16.814608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:16.814608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:16.814608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:16.814608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:16.814608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:16.814608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:16.814608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:16.814608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:16.814608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:16.814608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:16.814608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:16.814608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:16.815815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:16.815815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:16.815815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:16.815815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:16.815815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:16.815815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:16.815815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:16.815815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:16.815815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:16.815815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:16.815815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:19:16.815815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:16.815815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 73/146] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:16.834757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02767a38-ef75-4ee2-8085-aff96e8f4446 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:16.878201 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066bd8b-d7d2-4b40-b30a-3c1ca1a721c7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:16.933777 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe8ca5-eb37-4891-932a-6a43b4c4cf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:16.934590 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe8ca5-eb37-4891-932a-6a43b4c4cf84 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:16.934874 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe8ca5-eb37-4891-932a-6a43b4c4cf84 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:16.935154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe8ca5-eb37-4891-932a-6a43b4c4cf84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:16.950209 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe8ca5-eb37-4891-932a-6a43b4c4cf84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:16.962603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-0abe8ca5-eb37-4891-932a-6a43b4c4cf84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:19:16.962603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:16.962603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:16.962603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:16.962603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:16.962603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:16.962603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:16.962603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:16.962603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:16.962603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:16.962603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:16.962603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:16.962603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:16.962603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:16.962603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:16.962603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:16.964725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:16.964725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:16.964725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:16.964725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:16.964725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:16.964725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:16.964725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:16.964725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:16.964725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:16.964725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:16.964725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:16.964725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:16.964725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:16.964725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:16.964725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:16.964725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:16.966773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:16.966773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:16.966773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:16.966773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:16.966773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:16.966773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:16.966773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:16.966773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:16.966773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:16.966773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:16.966773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:19:16.966773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:16.966773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 74/147] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:19:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:16.970233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe8ca5-eb37-4891-932a-6a43b4c4cf84 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:16.992837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02767a38-ef75-4ee2-8085-aff96e8f4446 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:17.063554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e57ec-5e24-4c04-86a3-5906f21e3557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:17.064260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e57ec-5e24-4c04-86a3-5906f21e3557 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:17.064548 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e57ec-5e24-4c04-86a3-5906f21e3557 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:17.064825 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e57ec-5e24-4c04-86a3-5906f21e3557 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:17.065747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e57ec-5e24-4c04-86a3-5906f21e3557 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:17.066068 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e57ec-5e24-4c04-86a3-5906f21e3557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:17.067770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e57ec-5e24-4c04-86a3-5906f21e3557 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:17.083669 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:17.086261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 74/148] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:17.122879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e57ec-5e24-4c04-86a3-5906f21e3557 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:17.125620 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe8ca5-eb37-4891-932a-6a43b4c4cf84 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:17.165110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42fc32-7be5-483b-a7b9-b0dfcc21538f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:17.165847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42fc32-7be5-483b-a7b9-b0dfcc21538f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:17.166136 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42fc32-7be5-483b-a7b9-b0dfcc21538f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:17.166413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42fc32-7be5-483b-a7b9-b0dfcc21538f None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:17.167169 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42fc32-7be5-483b-a7b9-b0dfcc21538f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:17.167475 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42fc32-7be5-483b-a7b9-b0dfcc21538f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:17.168868 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42fc32-7be5-483b-a7b9-b0dfcc21538f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:17.184687 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 75/149] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:17.194967 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:17.195111 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42fc32-7be5-483b-a7b9-b0dfcc21538f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:17.271573 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e57ec-5e24-4c04-86a3-5906f21e3557 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:17.325438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf228a1-0511-4ef8-acbb-e49ca19e1609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:17.326395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf228a1-0511-4ef8-acbb-e49ca19e1609 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:17.326685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf228a1-0511-4ef8-acbb-e49ca19e1609 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:17.326966 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf228a1-0511-4ef8-acbb-e49ca19e1609 None admin] PATH_INFO: `/v3/users/1bb96419cbb049ee866966140d0ef35a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:17.331348 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf228a1-0511-4ef8-acbb-e49ca19e1609 None admin] RBAC: Authorizing `identity:get_user(user_id=1bb96419cbb049ee866966140d0ef35a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:17.332832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf228a1-0511-4ef8-acbb-e49ca19e1609 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:17.334870 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 75/150] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:17 2022] GET /identity/v3/users/1bb96419cbb049ee866966140d0ef35a => generated 289 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:17.344424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42fc32-7be5-483b-a7b9-b0dfcc21538f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:17.348148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf228a1-0511-4ef8-acbb-e49ca19e1609 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:17.393426 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:17.401773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00eccb-baf1-4b24-bd9b-d9e507d2c827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:17.408753 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00eccb-baf1-4b24-bd9b-d9e507d2c827 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:17.408753 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00eccb-baf1-4b24-bd9b-d9e507d2c827 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:17.408753 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00eccb-baf1-4b24-bd9b-d9e507d2c827 None admin] PATH_INFO: `/v3/users/4ce5a782a83f41fbb386c88e6d77ba55` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:17.408753 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00eccb-baf1-4b24-bd9b-d9e507d2c827 None admin] RBAC: Authorizing `identity:get_user(user_id=4ce5a782a83f41fbb386c88e6d77ba55)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:17.408753 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00eccb-baf1-4b24-bd9b-d9e507d2c827 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:17.408753 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 76/151] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:17 2022] GET /identity/v3/users/4ce5a782a83f41fbb386c88e6d77ba55 => generated 247 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:17.422363 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00eccb-baf1-4b24-bd9b-d9e507d2c827 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:17.497267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf228a1-0511-4ef8-acbb-e49ca19e1609 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:17.509484 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:17.548770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714929b9-cd35-4796-8bd6-1047de2a18de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:17.549843 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714929b9-cd35-4796-8bd6-1047de2a18de None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:17.550141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714929b9-cd35-4796-8bd6-1047de2a18de None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:17.550420 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714929b9-cd35-4796-8bd6-1047de2a18de None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:17.551707 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714929b9-cd35-4796-8bd6-1047de2a18de None admin] RBAC: Authorizing `identity:get_project(project_id=fe9516226a724cf688d3d23e1392161d)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:17.553438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714929b9-cd35-4796-8bd6-1047de2a18de None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:17.555501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 76/152] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:17 2022] GET /identity/v3/projects/fe9516226a724cf688d3d23e1392161d => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:17.563155 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a79864-b3f3-46e1-b1fe-077f168048c0 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:17.563441 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a79864-b3f3-46e1-b1fe-077f168048c0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:17.563722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a79864-b3f3-46e1-b1fe-077f168048c0 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:17.565328 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 77/153] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:19:17.573153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1165e6-5804-42e7-a2c6-3c7c71dfb691 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:17.573454 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1165e6-5804-42e7-a2c6-3c7c71dfb691 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:17.573721 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1165e6-5804-42e7-a2c6-3c7c71dfb691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:17.577256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00eccb-baf1-4b24-bd9b-d9e507d2c827 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:17.598962 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1f1165e6-5804-42e7-a2c6-3c7c71dfb691 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:17.600186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1f1165e6-5804-42e7-a2c6-3c7c71dfb691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:17.647260 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84fde6-fa75-49d8-823b-d424d285a4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:17.647955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84fde6-fa75-49d8-823b-d424d285a4ba None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:17.648329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84fde6-fa75-49d8-823b-d424d285a4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:17.648635 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84fde6-fa75-49d8-823b-d424d285a4ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:17.776263 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 78/154] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:17.777550 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:17.777658 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1165e6-5804-42e7-a2c6-3c7c71dfb691 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:17.781229 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84fde6-fa75-49d8-823b-d424d285a4ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:17.782727 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84fde6-fa75-49d8-823b-d424d285a4ba None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:17.984492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1165e6-5804-42e7-a2c6-3c7c71dfb691 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:17.989298 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 77/155] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:17.990056 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84fde6-fa75-49d8-823b-d424d285a4ba None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:17.997997 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:18.050841 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790d2bc-da9b-4b8c-9098-fd7aaa3e66e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:18.051751 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790d2bc-da9b-4b8c-9098-fd7aaa3e66e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:18.052193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790d2bc-da9b-4b8c-9098-fd7aaa3e66e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:18.052475 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790d2bc-da9b-4b8c-9098-fd7aaa3e66e1 None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d/users/1bb96419cbb049ee866966140d0ef35a/roles/5733a653e21248a7ab8ff7624b5f8ce9` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:18.058991 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790d2bc-da9b-4b8c-9098-fd7aaa3e66e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe9516226a724cf688d3d23e1392161d, user_id=1bb96419cbb049ee866966140d0ef35a, role_id=5733a653e21248a7ab8ff7624b5f8ce9)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:18.060480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790d2bc-da9b-4b8c-9098-fd7aaa3e66e1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:18.146782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 79/156] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:19:17 2022] PUT /identity/v3/projects/fe9516226a724cf688d3d23e1392161d/users/1bb96419cbb049ee866966140d0ef35a/roles/5733a653e21248a7ab8ff7624b5f8ce9 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:19:18.147605 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790d2bc-da9b-4b8c-9098-fd7aaa3e66e1 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:18.178212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84fde6-fa75-49d8-823b-d424d285a4ba None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:18.187787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:18.261950 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e13c7-ee55-42ac-9f8a-5df71b365614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:18.262672 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e13c7-ee55-42ac-9f8a-5df71b365614 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:18.262956 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e13c7-ee55-42ac-9f8a-5df71b365614 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:18.263230 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e13c7-ee55-42ac-9f8a-5df71b365614 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:18.294576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e13c7-ee55-42ac-9f8a-5df71b365614 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:18.302626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-f13e13c7-ee55-42ac-9f8a-5df71b365614 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:18.302626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:18.302626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:18.302626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:18.302626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:18.302626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:18.302626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:18.302626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:18.302626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:18.302626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:18.302626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:18.302626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:18.302626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:18.302626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:18.302626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:18.302626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:18.303532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:18.303532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:18.303532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:18.303532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:18.303532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:18.303532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:18.303532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:18.303532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:18.303532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:18.303532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:18.303532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:18.303532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:18.303532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:18.303532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:18.303532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:18.303532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:18.304718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:18.304718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:18.304718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:18.304718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:18.304718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:18.304718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:18.304718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:18.304718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:18.304718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:18.304718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:18.304718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:18.304718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:18.304718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 78/157] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:18.310124 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e13c7-ee55-42ac-9f8a-5df71b365614 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:18.388234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790d2bc-da9b-4b8c-9098-fd7aaa3e66e1 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:18.460935 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e13c7-ee55-42ac-9f8a-5df71b365614 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:18.501893 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96331837-0b8d-417b-85c9-b2e4d44fce4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:18.503594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96331837-0b8d-417b-85c9-b2e4d44fce4e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:18.504127 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96331837-0b8d-417b-85c9-b2e4d44fce4e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:18.504601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96331837-0b8d-417b-85c9-b2e4d44fce4e None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:18.506398 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96331837-0b8d-417b-85c9-b2e4d44fce4e None admin] RBAC: Authorizing `identity:get_project(project_id=fe9516226a724cf688d3d23e1392161d)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:18.508084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96331837-0b8d-417b-85c9-b2e4d44fce4e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:18.510423 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 80/158] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:18 2022] GET /identity/v3/projects/fe9516226a724cf688d3d23e1392161d => generated 298 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:18.512728 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13e644-235d-4251-a0ad-c87b3e684011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:18.513681 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13e644-235d-4251-a0ad-c87b3e684011 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:18.514163 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13e644-235d-4251-a0ad-c87b3e684011 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:18.515188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13e644-235d-4251-a0ad-c87b3e684011 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:18.516748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96331837-0b8d-417b-85c9-b2e4d44fce4e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:18.517882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13e644-235d-4251-a0ad-c87b3e684011 None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:18.520614 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13e644-235d-4251-a0ad-c87b3e684011 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:18.523088 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13e644-235d-4251-a0ad-c87b3e684011 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:18.541871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 79/159] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:18.547456 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13e644-235d-4251-a0ad-c87b3e684011 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:18.583856 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:18.601088 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:18.710060 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96331837-0b8d-417b-85c9-b2e4d44fce4e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:18.741363 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13e644-235d-4251-a0ad-c87b3e684011 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:18.771067 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737fe90-8b4b-4b1b-a9dd-e5ff50c57ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:18.772249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737fe90-8b4b-4b1b-a9dd-e5ff50c57ade None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:18.772793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737fe90-8b4b-4b1b-a9dd-e5ff50c57ade None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:18.773303 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737fe90-8b4b-4b1b-a9dd-e5ff50c57ade None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:18.774436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737fe90-8b4b-4b1b-a9dd-e5ff50c57ade None admin] RBAC: Adding query filter params (role.id=8c63c00418a346b29f7db3f87a404e70, scope.project.id=fe9516226a724cf688d3d23e1392161d, user.id=4ce5a782a83f41fbb386c88e6d77ba55) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:18.774950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737fe90-8b4b-4b1b-a9dd-e5ff50c57ade None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:18.776623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737fe90-8b4b-4b1b-a9dd-e5ff50c57ade None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:18.791480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 81/160] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:19:18 2022] GET /identity/v3/role_assignments?user.id=4ce5a782a83f41fbb386c88e6d77ba55&scope.project.id=fe9516226a724cf688d3d23e1392161d&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:18.835297 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a9b35-c699-4a39-b757-dd79e3a195fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:18.836224 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a9b35-c699-4a39-b757-dd79e3a195fc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:18.836514 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a9b35-c699-4a39-b757-dd79e3a195fc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:18.836795 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a9b35-c699-4a39-b757-dd79e3a195fc None admin] PATH_INFO: `/v3/users/73798cb88b144a3db46809da81f534cf` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:18.841768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a9b35-c699-4a39-b757-dd79e3a195fc None admin] RBAC: Authorizing `identity:get_user(user_id=73798cb88b144a3db46809da81f534cf)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:18.843315 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a9b35-c699-4a39-b757-dd79e3a195fc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:18.845232 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 80/161] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:18 2022] GET /identity/v3/users/73798cb88b144a3db46809da81f534cf => generated 275 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:18.850648 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737fe90-8b4b-4b1b-a9dd-e5ff50c57ade None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:18.988250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737fe90-8b4b-4b1b-a9dd-e5ff50c57ade None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:18.991510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:19.029599 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6adc6-c63b-450d-8bfc-0bd09cd9b0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:19.030210 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6adc6-c63b-450d-8bfc-0bd09cd9b0bf None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:19.030523 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6adc6-c63b-450d-8bfc-0bd09cd9b0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:19.030770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6adc6-c63b-450d-8bfc-0bd09cd9b0bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:19.106027 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd6adc6-c63b-450d-8bfc-0bd09cd9b0bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:19.109558 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd6adc6-c63b-450d-8bfc-0bd09cd9b0bf None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:19.227823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:19.232773 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 82/162] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:19.238230 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a9b35-c699-4a39-b757-dd79e3a195fc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:19.334037 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:19.343901 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a9b35-c699-4a39-b757-dd79e3a195fc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:19.379541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55728e46-b7a2-4a75-b3e6-e606cf00791d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:19.380407 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55728e46-b7a2-4a75-b3e6-e606cf00791d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:19.380688 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55728e46-b7a2-4a75-b3e6-e606cf00791d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:19.380963 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55728e46-b7a2-4a75-b3e6-e606cf00791d None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:19.382197 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55728e46-b7a2-4a75-b3e6-e606cf00791d None admin] RBAC: Authorizing `identity:get_project(project_id=fe9516226a724cf688d3d23e1392161d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:19.383640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55728e46-b7a2-4a75-b3e6-e606cf00791d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:19.385730 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 81/163] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:19 2022] GET /identity/v3/projects/fe9516226a724cf688d3d23e1392161d => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:19.394003 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6adc6-c63b-450d-8bfc-0bd09cd9b0bf None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:19.497520 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6adc6-c63b-450d-8bfc-0bd09cd9b0bf None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:19.513707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d7d79-24a8-44f3-8d24-69e1db7a12d8 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:19.513991 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d7d79-24a8-44f3-8d24-69e1db7a12d8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:19.514262 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d7d79-24a8-44f3-8d24-69e1db7a12d8 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:19.515658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 82/164] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:19.520888 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3ff10-c49f-4ad8-967c-5ac18d967dca None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:19.521199 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3ff10-c49f-4ad8-967c-5ac18d967dca None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:19.521477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3ff10-c49f-4ad8-967c-5ac18d967dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:19.540834 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2df3ff10-c49f-4ad8-967c-5ac18d967dca None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:19.542394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2df3ff10-c49f-4ad8-967c-5ac18d967dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:19.585960 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a497f-e9b3-4c90-a5f0-e40f5badb3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:19.586614 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a497f-e9b3-4c90-a5f0-e40f5badb3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:19.586847 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a497f-e9b3-4c90-a5f0-e40f5badb3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:19.587128 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a497f-e9b3-4c90-a5f0-e40f5badb3a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:19.587998 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a497f-e9b3-4c90-a5f0-e40f5badb3a4 None admin] RBAC: Adding query filter params (role.id=67111011e94c4e08928dd5e6b7a46af1, scope.project.id=fe9516226a724cf688d3d23e1392161d, user.id=73798cb88b144a3db46809da81f534cf) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:19.588315 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a497f-e9b3-4c90-a5f0-e40f5badb3a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:19.589860 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a497f-e9b3-4c90-a5f0-e40f5badb3a4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:19.615927 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 83/165] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:19:19 2022] GET /identity/v3/role_assignments?user.id=73798cb88b144a3db46809da81f534cf&scope.project.id=fe9516226a724cf688d3d23e1392161d&role.id=67111011e94c4e08928dd5e6b7a46af1 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:19.650370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 83/166] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:19.689538 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a497f-e9b3-4c90-a5f0-e40f5badb3a4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:19.767201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:19.777569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd8ba3-3713-4675-9889-1c35cda92160 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:19.777895 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd8ba3-3713-4675-9889-1c35cda92160 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:19.778131 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd8ba3-3713-4675-9889-1c35cda92160 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:19.779520 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 84/167] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:19.790729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2bf7a-3dfe-4786-b1ca-97a3ba230116 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:19.791035 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2bf7a-3dfe-4786-b1ca-97a3ba230116 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:19.791289 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2bf7a-3dfe-4786-b1ca-97a3ba230116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:19.821123 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7ae2bf7a-3dfe-4786-b1ca-97a3ba230116 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:19.822379 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7ae2bf7a-3dfe-4786-b1ca-97a3ba230116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:19.869772 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a497f-e9b3-4c90-a5f0-e40f5badb3a4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:19.947292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a0617-37f3-4630-8740-045aed7078a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:19.948102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a0617-37f3-4630-8740-045aed7078a0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:19.948383 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a0617-37f3-4630-8740-045aed7078a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:19.948663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a0617-37f3-4630-8740-045aed7078a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:19.965084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a0617-37f3-4630-8740-045aed7078a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:19.974492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-104a0617-37f3-4630-8740-045aed7078a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:19.974492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:19.974492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:19.974492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:19.974492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:19.974492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:19.974492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:19.974492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:19.974492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:19.974492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:19.974492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:19.974492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:19.974492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:19.974492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:19.974492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:19.974492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:19.975543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:19.975543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:19.975543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:19.975543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:19.975543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:19.975543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:19.975543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:19.975543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:19.975543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:19.975543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:19.975543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:19.975543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:19.975543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:19.975543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:19.975543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:19.975543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:19.976316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:19.976316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:19.976316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:19.976316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:19.976316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:19.976316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:19.976316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:19.976316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:19.976316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:19.976316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:19.976316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:19.976316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:19.976316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 84/168] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:19.983753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a0617-37f3-4630-8740-045aed7078a0 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:20.022399 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 85/169] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:20.053987 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2bf7a-3dfe-4786-b1ca-97a3ba230116 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:20.096549 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:20.127655 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a0617-37f3-4630-8740-045aed7078a0 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:20.163248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:20.194285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441ed80-9bf9-404d-b1a6-3ad663f7609b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:20.195145 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441ed80-9bf9-404d-b1a6-3ad663f7609b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:20.195434 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441ed80-9bf9-404d-b1a6-3ad663f7609b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:20.195711 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441ed80-9bf9-404d-b1a6-3ad663f7609b None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:20.196508 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441ed80-9bf9-404d-b1a6-3ad663f7609b None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:20.197279 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441ed80-9bf9-404d-b1a6-3ad663f7609b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:20.199117 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441ed80-9bf9-404d-b1a6-3ad663f7609b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:20.212516 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 85/170] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:20.224810 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441ed80-9bf9-404d-b1a6-3ad663f7609b None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:20.267313 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2bf7a-3dfe-4786-b1ca-97a3ba230116 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:20.371853 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3cd40-b80e-4e5e-9ab0-c674832ef1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:20.373200 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3cd40-b80e-4e5e-9ab0-c674832ef1db None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:20.373697 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3cd40-b80e-4e5e-9ab0-c674832ef1db None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:20.374769 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3cd40-b80e-4e5e-9ab0-c674832ef1db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:20.398284 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e3cd40-b80e-4e5e-9ab0-c674832ef1db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:20.407019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441ed80-9bf9-404d-b1a6-3ad663f7609b None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:20.413043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-b2e3cd40-b80e-4e5e-9ab0-c674832ef1db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:20.413043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:20.413043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:20.413043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:20.413043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:20.413043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:20.413043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:20.413043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:20.413043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:20.413043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:20.413043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:20.413043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:20.413043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:20.413043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:20.413043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:20.413043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:20.414011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:20.414011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:20.414011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:20.414011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:20.414011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:20.414011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:20.414011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:20.414011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:20.414011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:20.414011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:20.414011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:20.414011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:20.414011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:20.414011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:20.414011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:20.414011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:20.414775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:20.414775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:20.414775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:20.414775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:20.414775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:20.414775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:20.414775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:20.414775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:20.414775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:20.414775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:20.414775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:20.414775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:20.417898 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 86/171] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:19:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:20.423798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3cd40-b80e-4e5e-9ab0-c674832ef1db None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:20.469244 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b1778-95ac-4e2e-82d3-538b41ed410e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:20.470148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b1778-95ac-4e2e-82d3-538b41ed410e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:20.470427 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b1778-95ac-4e2e-82d3-538b41ed410e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:20.470698 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b1778-95ac-4e2e-82d3-538b41ed410e None admin] PATH_INFO: `/v3/users/73798cb88b144a3db46809da81f534cf` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:20.473330 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b1778-95ac-4e2e-82d3-538b41ed410e None admin] RBAC: Authorizing `identity:get_user(user_id=73798cb88b144a3db46809da81f534cf)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:20.473675 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b1778-95ac-4e2e-82d3-538b41ed410e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:20.475574 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 86/172] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:20 2022] GET /identity/v3/users/73798cb88b144a3db46809da81f534cf => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:20.484896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b1778-95ac-4e2e-82d3-538b41ed410e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:20.567521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:20.621320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:20.644302 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b1778-95ac-4e2e-82d3-538b41ed410e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:20.689299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3cd40-b80e-4e5e-9ab0-c674832ef1db None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:20.708281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3548152-e4cf-4bb7-9383-3ffa9b0f6dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:20.708948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3548152-e4cf-4bb7-9383-3ffa9b0f6dbc None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:20.709256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3548152-e4cf-4bb7-9383-3ffa9b0f6dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:20.709543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3548152-e4cf-4bb7-9383-3ffa9b0f6dbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:20.763963 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6dad0-539e-44a8-8ba8-0bce39af15ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:20.764670 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6dad0-539e-44a8-8ba8-0bce39af15ad None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:20.764958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6dad0-539e-44a8-8ba8-0bce39af15ad None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:20.766669 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6dad0-539e-44a8-8ba8-0bce39af15ad None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:20.767430 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6dad0-539e-44a8-8ba8-0bce39af15ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:20.767740 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6dad0-539e-44a8-8ba8-0bce39af15ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:20.769157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6dad0-539e-44a8-8ba8-0bce39af15ad None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:20.791000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 87/173] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:19:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:20.793329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30e36b-676f-4647-a264-5d87c211ccd9 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:20.793615 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30e36b-676f-4647-a264-5d87c211ccd9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:20.793891 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30e36b-676f-4647-a264-5d87c211ccd9 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:20.795117 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 88/174] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:20.801733 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30e36b-676f-4647-a264-5d87c211ccd9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:20.827248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3548152-e4cf-4bb7-9383-3ffa9b0f6dbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:20.828737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3548152-e4cf-4bb7-9383-3ffa9b0f6dbc None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:20.894823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:20.950452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30e36b-676f-4647-a264-5d87c211ccd9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:20.986409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 87/175] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 08:19:20.987504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97855f8a-c02b-4fed-b2a4-e95512c95618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:20.988298 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34678d91-cdc9-4011-9715-d06107a8151d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:20.988423 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97855f8a-c02b-4fed-b2a4-e95512c95618 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:20.988644 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97855f8a-c02b-4fed-b2a4-e95512c95618 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:20.988913 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97855f8a-c02b-4fed-b2a4-e95512c95618 None admin] PATH_INFO: `/v3/users/73798cb88b144a3db46809da81f534cf` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:20.989334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34678d91-cdc9-4011-9715-d06107a8151d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:20.989840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34678d91-cdc9-4011-9715-d06107a8151d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:20.990132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97855f8a-c02b-4fed-b2a4-e95512c95618 None admin] RBAC: Authorizing `identity:get_user(user_id=73798cb88b144a3db46809da81f534cf)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:20.991998 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97855f8a-c02b-4fed-b2a4-e95512c95618 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:20.993905 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 89/176] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:20 2022] GET /identity/v3/users/73798cb88b144a3db46809da81f534cf => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:20.995223 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37d1ef-c03a-440b-a701-bf9b19d14cc8 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:20.995515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37d1ef-c03a-440b-a701-bf9b19d14cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:20.995789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37d1ef-c03a-440b-a701-bf9b19d14cc8 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:21.001069 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 90/177] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:21.001069 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37d1ef-c03a-440b-a701-bf9b19d14cc8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:21.011756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:21.013218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-34678d91-cdc9-4011-9715-d06107a8151d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:21.014698 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-34678d91-cdc9-4011-9715-d06107a8151d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:21.101570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 88/178] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:21.102100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34678d91-cdc9-4011-9715-d06107a8151d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:21.121094 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37d1ef-c03a-440b-a701-bf9b19d14cc8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:21.160437 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb9850-42ec-467b-ab60-2ba1c3051253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:21.161725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb9850-42ec-467b-ab60-2ba1c3051253 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:21.162333 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb9850-42ec-467b-ab60-2ba1c3051253 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:21.162806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb9850-42ec-467b-ab60-2ba1c3051253 None admin] PATH_INFO: `/v3/projects/ebd04f5343964f4c9fc70331b9208d72` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:21.164294 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb9850-42ec-467b-ab60-2ba1c3051253 None admin] RBAC: Authorizing `identity:get_project(project_id=ebd04f5343964f4c9fc70331b9208d72)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:21.166040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb9850-42ec-467b-ab60-2ba1c3051253 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:21.167999 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 91/179] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:20 2022] GET /identity/v3/projects/ebd04f5343964f4c9fc70331b9208d72 => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:21.169382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ea95f-bef8-4213-b310-d84aa327bf23 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:21.169872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ea95f-bef8-4213-b310-d84aa327bf23 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:21.170405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ea95f-bef8-4213-b310-d84aa327bf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:21.188976 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9e7ea95f-bef8-4213-b310-d84aa327bf23 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:21.190693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9e7ea95f-bef8-4213-b310-d84aa327bf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:21.221091 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:21.241454 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34678d91-cdc9-4011-9715-d06107a8151d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:21.268140 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 92/180] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:21.277581 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ea95f-bef8-4213-b310-d84aa327bf23 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:21.278766 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4033c-3f3e-447f-b35e-5f6aba4e8a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:21.279423 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4033c-3f3e-447f-b35e-5f6aba4e8a07 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:21.279705 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4033c-3f3e-447f-b35e-5f6aba4e8a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:21.280180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4033c-3f3e-447f-b35e-5f6aba4e8a07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:21.287442 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:21.357522 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4033c-3f3e-447f-b35e-5f6aba4e8a07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:21.359269 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4033c-3f3e-447f-b35e-5f6aba4e8a07 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:21.401386 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ea95f-bef8-4213-b310-d84aa327bf23 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:21.438538 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6ecef-6134-4f36-9247-96379872d5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:21.439298 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6ecef-6134-4f36-9247-96379872d5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:21.439654 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6ecef-6134-4f36-9247-96379872d5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:21.439996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6ecef-6134-4f36-9247-96379872d5c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:21.441023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6ecef-6134-4f36-9247-96379872d5c2 None admin] RBAC: Adding query filter params (role.id=67111011e94c4e08928dd5e6b7a46af1, scope.project.id=ebd04f5343964f4c9fc70331b9208d72, user.id=73798cb88b144a3db46809da81f534cf) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:21.441451 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6ecef-6134-4f36-9247-96379872d5c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:21.442980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6ecef-6134-4f36-9247-96379872d5c2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:21.444469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:21.453121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 93/181] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:19:21 2022] GET /identity/v3/role_assignments?user.id=73798cb88b144a3db46809da81f534cf&scope.project.id=ebd04f5343964f4c9fc70331b9208d72&role.id=67111011e94c4e08928dd5e6b7a46af1 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:21.453121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6ecef-6134-4f36-9247-96379872d5c2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:21.478477 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 89/182] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:21.479882 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4033c-3f3e-447f-b35e-5f6aba4e8a07 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:21.536683 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:21.573778 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6ecef-6134-4f36-9247-96379872d5c2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:21.599246 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4033c-3f3e-447f-b35e-5f6aba4e8a07 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:21.612619 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26738feb-7093-42ba-aa3a-81b7fa71549d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:21.613589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26738feb-7093-42ba-aa3a-81b7fa71549d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:21.613923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26738feb-7093-42ba-aa3a-81b7fa71549d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:21.614277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26738feb-7093-42ba-aa3a-81b7fa71549d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:21.617260 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26738feb-7093-42ba-aa3a-81b7fa71549d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:21.618845 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26738feb-7093-42ba-aa3a-81b7fa71549d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:21.621169 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 94/183] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:19:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:21.622539 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da104e16-bb43-4f05-ba8f-b90dc4e3d211 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:21.622899 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da104e16-bb43-4f05-ba8f-b90dc4e3d211 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:21.623262 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da104e16-bb43-4f05-ba8f-b90dc4e3d211 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:21.625374 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 95/184] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:21.629572 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da104e16-bb43-4f05-ba8f-b90dc4e3d211 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:21.649333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336eeb8d-05bc-4f22-8d47-e848a9bf725b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:21.650305 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336eeb8d-05bc-4f22-8d47-e848a9bf725b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:21.650477 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336eeb8d-05bc-4f22-8d47-e848a9bf725b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:21.650729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336eeb8d-05bc-4f22-8d47-e848a9bf725b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:21.665474 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336eeb8d-05bc-4f22-8d47-e848a9bf725b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:21.675131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-336eeb8d-05bc-4f22-8d47-e848a9bf725b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:19:21.675131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:21.675131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:21.675131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:21.675131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:21.675131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:21.675131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:21.675131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:21.675131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:21.675131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:21.675131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:21.675131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:21.675131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:21.675131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:21.675131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:21.675131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:21.675995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:21.675995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:21.675995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:21.675995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:21.675995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:21.675995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:21.675995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:21.675995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:21.675995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:21.675995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:21.675995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:21.675995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:21.675995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:21.675995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:21.675995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:21.675995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:21.676760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:21.676760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:21.676760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:21.676760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:21.676760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:21.676760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:21.676760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:21.676760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:21.676760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:21.676760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:21.676760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:19:21.676760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:21.676760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 90/185] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:19:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:21.676760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336eeb8d-05bc-4f22-8d47-e848a9bf725b None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:21.733820 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:21.757796 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da104e16-bb43-4f05-ba8f-b90dc4e3d211 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:21.799937 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e80bf-c51f-46a2-aac0-157db6999a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:21.800819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e80bf-c51f-46a2-aac0-157db6999a15 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:21.801124 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e80bf-c51f-46a2-aac0-157db6999a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:21.801406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e80bf-c51f-46a2-aac0-157db6999a15 None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:21.802586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e80bf-c51f-46a2-aac0-157db6999a15 None admin] RBAC: Authorizing `identity:get_project(project_id=fe9516226a724cf688d3d23e1392161d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:21.804026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e80bf-c51f-46a2-aac0-157db6999a15 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:21.805741 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 96/186] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:21 2022] GET /identity/v3/projects/fe9516226a724cf688d3d23e1392161d => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:21.806876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032659a-482f-49c8-83d5-5bb9fafd27d6 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:21.807169 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032659a-482f-49c8-83d5-5bb9fafd27d6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:21.807433 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032659a-482f-49c8-83d5-5bb9fafd27d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:21.810739 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336eeb8d-05bc-4f22-8d47-e848a9bf725b None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:21.818778 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:21.824254 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6032659a-482f-49c8-83d5-5bb9fafd27d6 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:21.825476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6032659a-482f-49c8-83d5-5bb9fafd27d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:21.844347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e3d66-ff2e-45ec-ae90-55a984c3db9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:21.845025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e3d66-ff2e-45ec-ae90-55a984c3db9d None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:21.845321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e3d66-ff2e-45ec-ae90-55a984c3db9d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:21.845598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e3d66-ff2e-45ec-ae90-55a984c3db9d None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:21.846302 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e3d66-ff2e-45ec-ae90-55a984c3db9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:21.847742 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e3d66-ff2e-45ec-ae90-55a984c3db9d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:21.891715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 97/187] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:21.892543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032659a-482f-49c8-83d5-5bb9fafd27d6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:21.917583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 91/188] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jun 7 08:19:21 2022] POST /identity/v3/users => generated 297 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:19:21.919110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ecd7a-0254-4440-9008-78a51f482ea0 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:21.919404 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ecd7a-0254-4440-9008-78a51f482ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:21.919689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ecd7a-0254-4440-9008-78a51f482ea0 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:21.921584 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 92/189] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:21.921584 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ecd7a-0254-4440-9008-78a51f482ea0 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:22.020828 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032659a-482f-49c8-83d5-5bb9fafd27d6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:22.032769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:22.039017 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ecd7a-0254-4440-9008-78a51f482ea0 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:22.060848 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5e9ad-97e9-44aa-8616-45b1437f2763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:22.062247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5e9ad-97e9-44aa-8616-45b1437f2763 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:22.062589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5e9ad-97e9-44aa-8616-45b1437f2763 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:22.063238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5e9ad-97e9-44aa-8616-45b1437f2763 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:22.063639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5e9ad-97e9-44aa-8616-45b1437f2763 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:22.063950 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5e9ad-97e9-44aa-8616-45b1437f2763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:22.065594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5e9ad-97e9-44aa-8616-45b1437f2763 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:22.068597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:22.075599 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 98/190] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:22.079262 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67499548-b1db-461e-ab3c-540e9d9256a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:22.079936 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67499548-b1db-461e-ab3c-540e9d9256a4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:22.080381 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67499548-b1db-461e-ab3c-540e9d9256a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:22.080565 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67499548-b1db-461e-ab3c-540e9d9256a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:22.081163 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5e9ad-97e9-44aa-8616-45b1437f2763 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:22.081458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67499548-b1db-461e-ab3c-540e9d9256a4 None admin] RBAC: Adding query filter params (role.id=4552e69b96eb43f3ba6a9ca694d04ffb, scope.project.id=fe9516226a724cf688d3d23e1392161d, user.id=73798cb88b144a3db46809da81f534cf) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:22.081777 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67499548-b1db-461e-ab3c-540e9d9256a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:22.083260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67499548-b1db-461e-ab3c-540e9d9256a4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:22.089974 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 93/191] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:19:21 2022] GET /identity/v3/role_assignments?user.id=73798cb88b144a3db46809da81f534cf&scope.project.id=fe9516226a724cf688d3d23e1392161d&role.id=4552e69b96eb43f3ba6a9ca694d04ffb => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:22.091284 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bcd33-3e2e-49e7-87c8-ef353b7697c6 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:22.091571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bcd33-3e2e-49e7-87c8-ef353b7697c6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:22.091848 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bcd33-3e2e-49e7-87c8-ef353b7697c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:22.111751 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-140bcd33-3e2e-49e7-87c8-ef353b7697c6 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:22.114256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-140bcd33-3e2e-49e7-87c8-ef353b7697c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:22.216450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 94/192] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:22.217419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bcd33-3e2e-49e7-87c8-ef353b7697c6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:22.257016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5e9ad-97e9-44aa-8616-45b1437f2763 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:22.297624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc622d-08ec-467d-a3cb-3d77b9e36702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:22.298370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc622d-08ec-467d-a3cb-3d77b9e36702 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:22.298654 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc622d-08ec-467d-a3cb-3d77b9e36702 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:22.298929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc622d-08ec-467d-a3cb-3d77b9e36702 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:22.311888 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc622d-08ec-467d-a3cb-3d77b9e36702 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:22.320495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-cecc622d-08ec-467d-a3cb-3d77b9e36702 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:22.320495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:22.320495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:22.320495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:22.320495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:22.320495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:22.320495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:22.320495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:22.320495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:22.320495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:22.320495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:22.320495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:22.320495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:22.320495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:22.320495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:22.320495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:22.321494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:22.321494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:22.321494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:22.321494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:22.321494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:22.321494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:22.321494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:22.321494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:22.321494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:22.321494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:22.321494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:22.321494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:22.321494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:22.321494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:22.321494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:22.321494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:22.322303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:22.322303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:22.322303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:22.322303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:22.322303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:22.322303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:22.322303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:22.322303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:22.322303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:22.322303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:22.322303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:22.322303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:22.322303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 99/193] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:19:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:22.323573 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc622d-08ec-467d-a3cb-3d77b9e36702 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:22.331794 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:22.341455 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bcd33-3e2e-49e7-87c8-ef353b7697c6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:22.394367 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b2861-f4e6-416c-88d9-ede24b2fc355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:22.396286 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b2861-f4e6-416c-88d9-ede24b2fc355 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:22.397063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b2861-f4e6-416c-88d9-ede24b2fc355 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:22.397551 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b2861-f4e6-416c-88d9-ede24b2fc355 None admin] PATH_INFO: `/v3/users/5d1e906881c5465687c5968efcae625e` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:22.398221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:22.417316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b2861-f4e6-416c-88d9-ede24b2fc355 None admin] RBAC: Authorizing `identity:get_user(user_id=5d1e906881c5465687c5968efcae625e)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:22.419894 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b2861-f4e6-416c-88d9-ede24b2fc355 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:22.422891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 95/194] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:22 2022] GET /identity/v3/users/5d1e906881c5465687c5968efcae625e => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:22.424425 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b2861-f4e6-416c-88d9-ede24b2fc355 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:22.443767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc622d-08ec-467d-a3cb-3d77b9e36702 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:22.481320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a396ffd-96d4-40e8-abbf-e1d724549ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:22.482128 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a396ffd-96d4-40e8-abbf-e1d724549ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:22.482413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a396ffd-96d4-40e8-abbf-e1d724549ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:22.482692 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a396ffd-96d4-40e8-abbf-e1d724549ad6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:22.494848 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a396ffd-96d4-40e8-abbf-e1d724549ad6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:22.504350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8a396ffd-96d4-40e8-abbf-e1d724549ad6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:19:22.504350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:22.504350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:22.504350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:22.504350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:22.504350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:22.504350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:22.504350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:22.504350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:22.504350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:22.504350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:22.504350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:22.504350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:22.504350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:22.504350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:22.504350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:22.505264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:22.505264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:22.505264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:22.505264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:22.505264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:22.505264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:22.505264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:22.505264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:22.505264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:22.505264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:22.505264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:22.505264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:22.505264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:22.505264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:22.505264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:22.505264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:22.506040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:22.506040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:22.506040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:22.506040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:22.506040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:22.506040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:22.506040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:22.506040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:22.506040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:22.506040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:22.506040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:19:22.506040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:22.507175 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 100/195] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:22.507844 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a396ffd-96d4-40e8-abbf-e1d724549ad6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:22.563550 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b2861-f4e6-416c-88d9-ede24b2fc355 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:22.603852 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da986c5-02db-4ff4-8f6a-19376f443061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:22.604527 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da986c5-02db-4ff4-8f6a-19376f443061 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:22.604814 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da986c5-02db-4ff4-8f6a-19376f443061 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:22.605133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da986c5-02db-4ff4-8f6a-19376f443061 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:22.605894 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da986c5-02db-4ff4-8f6a-19376f443061 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:22.606210 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da986c5-02db-4ff4-8f6a-19376f443061 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:22.607596 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da986c5-02db-4ff4-8f6a-19376f443061 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:22.617499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 96/196] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:19:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:22.618121 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da986c5-02db-4ff4-8f6a-19376f443061 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:22.636382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:22.651297 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a396ffd-96d4-40e8-abbf-e1d724549ad6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:22.679164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:22.704385 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1a701-4cfa-49eb-8281-233d001fd2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:22.705570 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1a701-4cfa-49eb-8281-233d001fd2af None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:22.706072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1a701-4cfa-49eb-8281-233d001fd2af None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:22.706536 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1a701-4cfa-49eb-8281-233d001fd2af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:22.752526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da986c5-02db-4ff4-8f6a-19376f443061 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:22.781657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1a701-4cfa-49eb-8281-233d001fd2af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:22.783766 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1a701-4cfa-49eb-8281-233d001fd2af None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:22.795625 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97466204-c4a0-4d80-a120-3fed4340743c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:22.796326 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97466204-c4a0-4d80-a120-3fed4340743c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:22.796611 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97466204-c4a0-4d80-a120-3fed4340743c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:22.797245 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97466204-c4a0-4d80-a120-3fed4340743c None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:22.798064 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97466204-c4a0-4d80-a120-3fed4340743c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:22.798388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97466204-c4a0-4d80-a120-3fed4340743c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:22.800326 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97466204-c4a0-4d80-a120-3fed4340743c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:22.811038 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 97/197] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:22.813366 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97466204-c4a0-4d80-a120-3fed4340743c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:22.899961 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:22.926251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 101/198] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:22.927114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1a701-4cfa-49eb-8281-233d001fd2af None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:22.927728 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97466204-c4a0-4d80-a120-3fed4340743c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:22.969366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:22.977367 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736bdef-b341-4c96-abe2-572694eeb611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:22.978259 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736bdef-b341-4c96-abe2-572694eeb611 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:22.978546 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736bdef-b341-4c96-abe2-572694eeb611 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:22.979188 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736bdef-b341-4c96-abe2-572694eeb611 None admin] PATH_INFO: `/v3/users/5d1e906881c5465687c5968efcae625e` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:22.980773 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736bdef-b341-4c96-abe2-572694eeb611 None admin] RBAC: Authorizing `identity:get_user(user_id=5d1e906881c5465687c5968efcae625e)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:22.982451 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736bdef-b341-4c96-abe2-572694eeb611 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:22.985054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 98/199] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:22 2022] GET /identity/v3/users/5d1e906881c5465687c5968efcae625e => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:22.985677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736bdef-b341-4c96-abe2-572694eeb611 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:23.042521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1a701-4cfa-49eb-8281-233d001fd2af None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:23.081343 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4ce6b-e5d8-4045-b17e-9289c3d72dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:23.082533 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4ce6b-e5d8-4045-b17e-9289c3d72dee None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:23.083006 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4ce6b-e5d8-4045-b17e-9289c3d72dee None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:23.083514 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4ce6b-e5d8-4045-b17e-9289c3d72dee None admin] PATH_INFO: `/v3/users/1bb96419cbb049ee866966140d0ef35a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:23.084880 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce4ce6b-e5d8-4045-b17e-9289c3d72dee None admin] RBAC: Authorizing `identity:get_user(user_id=1bb96419cbb049ee866966140d0ef35a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:23.086684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce4ce6b-e5d8-4045-b17e-9289c3d72dee None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:23.088593 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 102/200] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:22 2022] GET /identity/v3/users/1bb96419cbb049ee866966140d0ef35a => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:23.089688 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4ce6b-e5d8-4045-b17e-9289c3d72dee None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:23.122408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736bdef-b341-4c96-abe2-572694eeb611 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:23.161763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad353ace-db77-4e31-9307-17f1b65955c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:23.162509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad353ace-db77-4e31-9307-17f1b65955c1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:23.162791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad353ace-db77-4e31-9307-17f1b65955c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:23.163072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad353ace-db77-4e31-9307-17f1b65955c1 None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:23.164267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad353ace-db77-4e31-9307-17f1b65955c1 None admin] RBAC: Authorizing `identity:get_project(project_id=914fc5f0eaf745279782c417df7acf69)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:23.165984 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad353ace-db77-4e31-9307-17f1b65955c1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:23.168008 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 99/201] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:22 2022] GET /identity/v3/projects/914fc5f0eaf745279782c417df7acf69 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:23.168592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad353ace-db77-4e31-9307-17f1b65955c1 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:23.188633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:23.197739 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:23.213714 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4ce6b-e5d8-4045-b17e-9289c3d72dee None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:23.251657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad2fde-d491-47e7-aeec-a4c72364567e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:23.252645 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad2fde-d491-47e7-aeec-a4c72364567e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:23.253298 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad2fde-d491-47e7-aeec-a4c72364567e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:23.253881 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad2fde-d491-47e7-aeec-a4c72364567e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:23.292555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad353ace-db77-4e31-9307-17f1b65955c1 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:23.333695 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e1364-9745-469d-a3b0-9b24380a5160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:23.334707 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e1364-9745-469d-a3b0-9b24380a5160 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:23.335266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e1364-9745-469d-a3b0-9b24380a5160 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:23.335740 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e1364-9745-469d-a3b0-9b24380a5160 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:23.340735 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad2fde-d491-47e7-aeec-a4c72364567e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:23.342229 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad2fde-d491-47e7-aeec-a4c72364567e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:23.422353 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e1364-9745-469d-a3b0-9b24380a5160 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:23.423843 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e1364-9745-469d-a3b0-9b24380a5160 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:23.443010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:23.453407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:23.462609 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 103/202] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:23.465539 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad2fde-d491-47e7-aeec-a4c72364567e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:23.529282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 100/203] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:23.530099 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e1364-9745-469d-a3b0-9b24380a5160 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:23.586620 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad2fde-d491-47e7-aeec-a4c72364567e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:23.622804 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1431dbf-69ff-41d5-aa2f-75cc717ffa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:23.623553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1431dbf-69ff-41d5-aa2f-75cc717ffa87 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:23.624362 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1431dbf-69ff-41d5-aa2f-75cc717ffa87 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:23.624645 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1431dbf-69ff-41d5-aa2f-75cc717ffa87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:23.625501 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1431dbf-69ff-41d5-aa2f-75cc717ffa87 None admin] RBAC: Adding query filter params (role.id=8c63c00418a346b29f7db3f87a404e70, scope.project.id=914fc5f0eaf745279782c417df7acf69, user.id=5d1e906881c5465687c5968efcae625e) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:23.625908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1431dbf-69ff-41d5-aa2f-75cc717ffa87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:23.627439 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1431dbf-69ff-41d5-aa2f-75cc717ffa87 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:23.635411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 104/204] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:19:23 2022] GET /identity/v3/role_assignments?user.id=5d1e906881c5465687c5968efcae625e&scope.project.id=914fc5f0eaf745279782c417df7acf69&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:23.636211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1431dbf-69ff-41d5-aa2f-75cc717ffa87 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:23.650433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e1364-9745-469d-a3b0-9b24380a5160 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:23.663515 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:23.694464 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ade108-c48e-402e-98d3-4f99dde5ef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:23.695281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ade108-c48e-402e-98d3-4f99dde5ef00 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:23.695844 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ade108-c48e-402e-98d3-4f99dde5ef00 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:23.696187 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ade108-c48e-402e-98d3-4f99dde5ef00 None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:23.698668 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ade108-c48e-402e-98d3-4f99dde5ef00 None admin] RBAC: Authorizing `identity:get_project(project_id=914fc5f0eaf745279782c417df7acf69)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:23.700178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ade108-c48e-402e-98d3-4f99dde5ef00 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:23.702044 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 101/205] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:23 2022] GET /identity/v3/projects/914fc5f0eaf745279782c417df7acf69 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:23.703280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:23.709299 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ade108-c48e-402e-98d3-4f99dde5ef00 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:23.805140 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1431dbf-69ff-41d5-aa2f-75cc717ffa87 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:23.874334 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab030be-16f7-4037-a1fd-5114f838ed5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:23.875304 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab030be-16f7-4037-a1fd-5114f838ed5d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:23.875588 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab030be-16f7-4037-a1fd-5114f838ed5d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:23.875892 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab030be-16f7-4037-a1fd-5114f838ed5d None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:23.878966 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab030be-16f7-4037-a1fd-5114f838ed5d None admin] RBAC: Authorizing `identity:get_project(project_id=fe9516226a724cf688d3d23e1392161d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:23.880540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab030be-16f7-4037-a1fd-5114f838ed5d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:23.882590 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 105/206] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:23 2022] GET /identity/v3/projects/fe9516226a724cf688d3d23e1392161d => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:23.888279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab030be-16f7-4037-a1fd-5114f838ed5d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:23.889592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ade108-c48e-402e-98d3-4f99dde5ef00 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:23.928319 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd021df8-0146-40c6-947a-c0250411270d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:23.929387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd021df8-0146-40c6-947a-c0250411270d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:23.929840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd021df8-0146-40c6-947a-c0250411270d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:23.930332 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd021df8-0146-40c6-947a-c0250411270d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:23.931408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd021df8-0146-40c6-947a-c0250411270d None admin] RBAC: Adding query filter params (role.id=4552e69b96eb43f3ba6a9ca694d04ffb, scope.project.id=914fc5f0eaf745279782c417df7acf69, user.id=5d1e906881c5465687c5968efcae625e) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:23.931930 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd021df8-0146-40c6-947a-c0250411270d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:23.933614 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd021df8-0146-40c6-947a-c0250411270d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:23.940661 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 102/207] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:19:23 2022] GET /identity/v3/role_assignments?user.id=5d1e906881c5465687c5968efcae625e&scope.project.id=914fc5f0eaf745279782c417df7acf69&role.id=4552e69b96eb43f3ba6a9ca694d04ffb => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:24.023598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab030be-16f7-4037-a1fd-5114f838ed5d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:24.043439 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:24.063422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04166726-e505-4b5d-a4eb-d7552f6d95f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:24.064549 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04166726-e505-4b5d-a4eb-d7552f6d95f7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:24.064751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04166726-e505-4b5d-a4eb-d7552f6d95f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:24.065025 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04166726-e505-4b5d-a4eb-d7552f6d95f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:24.065919 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04166726-e505-4b5d-a4eb-d7552f6d95f7 None admin] RBAC: Adding query filter params (role.id=5733a653e21248a7ab8ff7624b5f8ce9, scope.project.id=fe9516226a724cf688d3d23e1392161d, user.id=1bb96419cbb049ee866966140d0ef35a) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:24.066237 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04166726-e505-4b5d-a4eb-d7552f6d95f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:24.067679 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04166726-e505-4b5d-a4eb-d7552f6d95f7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:24.074230 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 106/208] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:19:23 2022] GET /identity/v3/role_assignments?user.id=1bb96419cbb049ee866966140d0ef35a&scope.project.id=fe9516226a724cf688d3d23e1392161d&role.id=5733a653e21248a7ab8ff7624b5f8ce9 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:24.556287 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b7a0e-a9fe-4b99-a18f-8e7a1b873394 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:24.556683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b7a0e-a9fe-4b99-a18f-8e7a1b873394 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:24.556884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b7a0e-a9fe-4b99-a18f-8e7a1b873394 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:24.558399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 103/209] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:24.558685 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7844dc0-97de-4532-9479-2b789ced4a4a None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:24.558971 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7844dc0-97de-4532-9479-2b789ced4a4a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:24.559250 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7844dc0-97de-4532-9479-2b789ced4a4a None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:24.560616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 107/210] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:24.565340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f0283-c9f7-430a-9185-f273e55608a6 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:24.565623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f0283-c9f7-430a-9185-f273e55608a6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:24.565892 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f0283-c9f7-430a-9185-f273e55608a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:24.566393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3109e2f-25c3-4d8a-9a0d-e925a1ce9070 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:24.566680 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3109e2f-25c3-4d8a-9a0d-e925a1ce9070 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:24.566957 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3109e2f-25c3-4d8a-9a0d-e925a1ce9070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:24.585542 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c3109e2f-25c3-4d8a-9a0d-e925a1ce9070 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:24.586728 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c3109e2f-25c3-4d8a-9a0d-e925a1ce9070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:24.588689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ff2f0283-c9f7-430a-9185-f273e55608a6 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:24.589877 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ff2f0283-c9f7-430a-9185-f273e55608a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:24.634796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:24.662241 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 108/211] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:24.682152 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 104/212] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:24.688441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3109e2f-25c3-4d8a-9a0d-e925a1ce9070 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:24.709176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f0283-c9f7-430a-9185-f273e55608a6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:24.801816 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3109e2f-25c3-4d8a-9a0d-e925a1ce9070 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:24.816046 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:24.823749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f0283-c9f7-430a-9185-f273e55608a6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:24.838302 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13125d-7173-481c-9679-2628a24f6e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:24.838302 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13125d-7173-481c-9679-2628a24f6e4a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:24.838700 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13125d-7173-481c-9679-2628a24f6e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:24.838877 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13125d-7173-481c-9679-2628a24f6e4a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:24.840088 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc13125d-7173-481c-9679-2628a24f6e4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:24.841752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc13125d-7173-481c-9679-2628a24f6e4a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:24.847815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 109/213] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:19:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:24.848909 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13125d-7173-481c-9679-2628a24f6e4a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:24.867102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b4d1a-55cb-4e81-b67b-e157f6d0f2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:24.868246 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b4d1a-55cb-4e81-b67b-e157f6d0f2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:24.868682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b4d1a-55cb-4e81-b67b-e157f6d0f2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:24.869173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b4d1a-55cb-4e81-b67b-e157f6d0f2f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:24.880781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b4d1a-55cb-4e81-b67b-e157f6d0f2f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:24.889035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-ce8b4d1a-55cb-4e81-b67b-e157f6d0f2f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:24.889035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:24.889035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:24.889035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:24.889035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:24.889035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:24.889035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:24.889035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:24.889035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:24.889035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:24.889035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:24.889035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:24.889035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:24.889035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:24.889035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:24.889035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:24.890408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:24.890408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:24.890408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:24.890408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:24.890408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:24.890408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:24.890408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:24.890408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:24.890408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:24.890408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:24.890408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:24.890408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:24.890408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:24.890408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:24.890408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:24.890408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:24.891178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:24.891178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:24.891178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:24.891178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:24.891178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:24.891178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:24.891178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:24.891178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:24.891178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:24.891178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:24.891178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:24.891178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:24.893994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 105/214] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:19:24.899345 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b4d1a-55cb-4e81-b67b-e157f6d0f2f4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:24.932271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:24.971537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13125d-7173-481c-9679-2628a24f6e4a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:25.003944 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b4d1a-55cb-4e81-b67b-e157f6d0f2f4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:25.015837 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8d761-6ba7-41f7-a8e1-4650db75618e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:25.016778 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8d761-6ba7-41f7-a8e1-4650db75618e None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:25.017466 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8d761-6ba7-41f7-a8e1-4650db75618e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:25.017963 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8d761-6ba7-41f7-a8e1-4650db75618e None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:25.019448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8d761-6ba7-41f7-a8e1-4650db75618e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:25.021287 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8d761-6ba7-41f7-a8e1-4650db75618e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:25.038970 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b58bcc-054f-421c-825f-673667ac35e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:25.039710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b58bcc-054f-421c-825f-673667ac35e2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:25.040086 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b58bcc-054f-421c-825f-673667ac35e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:25.040431 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b58bcc-054f-421c-825f-673667ac35e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:25.041281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b58bcc-054f-421c-825f-673667ac35e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:25.041703 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b58bcc-054f-421c-825f-673667ac35e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:25.043093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b58bcc-054f-421c-825f-673667ac35e2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:25.052481 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 106/215] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:25.057665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b58bcc-054f-421c-825f-673667ac35e2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:25.130023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 110/216] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jun 7 08:19:24 2022] POST /identity/v3/users => generated 297 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:19:25.177747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:25.181477 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b58bcc-054f-421c-825f-673667ac35e2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:25.221334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e47213-5c2b-454a-85f8-b5d9c9126a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:25.222234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e47213-5c2b-454a-85f8-b5d9c9126a96 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:25.222518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e47213-5c2b-454a-85f8-b5d9c9126a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:25.222797 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e47213-5c2b-454a-85f8-b5d9c9126a96 None admin] PATH_INFO: `/v3/users/d7d3784ac4294257add25142da9d8de4` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:25.239306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e47213-5c2b-454a-85f8-b5d9c9126a96 None admin] RBAC: Authorizing `identity:get_user(user_id=d7d3784ac4294257add25142da9d8de4)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:25.240923 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e47213-5c2b-454a-85f8-b5d9c9126a96 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:25.242712 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 107/217] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:25 2022] GET /identity/v3/users/d7d3784ac4294257add25142da9d8de4 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:25.248602 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8d761-6ba7-41f7-a8e1-4650db75618e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:25.293131 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:25.368565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8d761-6ba7-41f7-a8e1-4650db75618e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:25.403374 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c626fcd-9b9b-4ac6-bc33-b739775f5507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:25.404349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c626fcd-9b9b-4ac6-bc33-b739775f5507 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:25.404829 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c626fcd-9b9b-4ac6-bc33-b739775f5507 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:25.405838 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c626fcd-9b9b-4ac6-bc33-b739775f5507 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:25.476342 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c626fcd-9b9b-4ac6-bc33-b739775f5507 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:25.478188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c626fcd-9b9b-4ac6-bc33-b739775f5507 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:25.516283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:25.585036 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 111/218] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:25.590571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e47213-5c2b-454a-85f8-b5d9c9126a96 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:25.698735 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e47213-5c2b-454a-85f8-b5d9c9126a96 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:25.734685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900ba44-c6be-478e-9cfa-49335d34502c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:25.735779 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900ba44-c6be-478e-9cfa-49335d34502c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:25.736224 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900ba44-c6be-478e-9cfa-49335d34502c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:25.736633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900ba44-c6be-478e-9cfa-49335d34502c None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:25.737972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900ba44-c6be-478e-9cfa-49335d34502c None admin] RBAC: Authorizing `identity:get_project(project_id=914fc5f0eaf745279782c417df7acf69)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:25.739569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900ba44-c6be-478e-9cfa-49335d34502c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:25.741439 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 108/219] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:25 2022] GET /identity/v3/projects/914fc5f0eaf745279782c417df7acf69 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:25.747386 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c626fcd-9b9b-4ac6-bc33-b739775f5507 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:25.874808 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c626fcd-9b9b-4ac6-bc33-b739775f5507 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:25.926022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a177ad-4858-42af-8023-dc905f592091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:25.926967 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a177ad-4858-42af-8023-dc905f592091 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:25.927389 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a177ad-4858-42af-8023-dc905f592091 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:25.927799 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a177ad-4858-42af-8023-dc905f592091 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:25.928783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a177ad-4858-42af-8023-dc905f592091 None admin] RBAC: Adding query filter params (role.id=67111011e94c4e08928dd5e6b7a46af1, scope.project.id=914fc5f0eaf745279782c417df7acf69, user.id=d7d3784ac4294257add25142da9d8de4) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:25.929256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a177ad-4858-42af-8023-dc905f592091 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:25.930853 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a177ad-4858-42af-8023-dc905f592091 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:25.938334 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:25.942478 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 112/220] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:19:25 2022] GET /identity/v3/role_assignments?user.id=d7d3784ac4294257add25142da9d8de4&scope.project.id=914fc5f0eaf745279782c417df7acf69&role.id=67111011e94c4e08928dd5e6b7a46af1 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:26.268443 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15205ff-35fe-4446-aaaf-687484d81ed6 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:26.268950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15205ff-35fe-4446-aaaf-687484d81ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:26.269141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15205ff-35fe-4446-aaaf-687484d81ed6 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:26.270460 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 109/221] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:26.276117 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09869697-4caf-47b9-8664-cb5820afcc6f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:26.276409 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09869697-4caf-47b9-8664-cb5820afcc6f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:26.276693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09869697-4caf-47b9-8664-cb5820afcc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:26.302048 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-09869697-4caf-47b9-8664-cb5820afcc6f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:26.303251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-09869697-4caf-47b9-8664-cb5820afcc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:26.398058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 113/222] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:26.432500 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15205ff-35fe-4446-aaaf-687484d81ed6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:26.467378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:26.553822 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266862ad-879c-46af-8722-fd6dce92508f None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:26.554146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266862ad-879c-46af-8722-fd6dce92508f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:26.554389 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266862ad-879c-46af-8722-fd6dce92508f None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:26.556000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 114/223] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:26.561971 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d97b9f-e6d1-456b-bdd1-121671bc05a3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:26.562205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d97b9f-e6d1-456b-bdd1-121671bc05a3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:26.562488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d97b9f-e6d1-456b-bdd1-121671bc05a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:26.591693 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15205ff-35fe-4446-aaaf-687484d81ed6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:26.596415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-72d97b9f-e6d1-456b-bdd1-121671bc05a3 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:26.597819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-72d97b9f-e6d1-456b-bdd1-121671bc05a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:26.652591 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4d814-cab4-4d08-87de-4b06be054024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:26.653730 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4d814-cab4-4d08-87de-4b06be054024 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:26.654183 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4d814-cab4-4d08-87de-4b06be054024 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:26.654625 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4d814-cab4-4d08-87de-4b06be054024 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:26.672938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4d814-cab4-4d08-87de-4b06be054024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:26.681938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-89c4d814-cab4-4d08-87de-4b06be054024 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:19:26.681938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:26.681938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:26.681938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:26.681938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:26.681938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:26.681938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:26.681938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:26.681938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:26.681938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:26.681938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:26.681938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:26.681938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:26.681938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:26.681938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:26.681938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:26.682784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:26.682784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:26.682784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:26.682784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:26.682784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:26.682784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:26.682784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:26.682784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:26.682784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:26.682784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:26.682784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:26.682784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:26.682784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:26.682784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:26.682784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:26.682784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:26.683554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:26.683554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:26.683554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:26.683554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:26.683554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:26.683554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:26.683554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:26.683554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:26.683554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:26.683554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:26.683554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:19:26.683554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:26.685960 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 110/224] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:19:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:26.691759 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4d814-cab4-4d08-87de-4b06be054024 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:26.734780 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 115/225] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:26.769622 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d97b9f-e6d1-456b-bdd1-121671bc05a3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:26.843658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:26.865438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:26.887023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d97b9f-e6d1-456b-bdd1-121671bc05a3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:26.923938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655b01e-a7a6-47bd-a82f-6e4378a9a571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:26.925415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655b01e-a7a6-47bd-a82f-6e4378a9a571 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:26.925918 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655b01e-a7a6-47bd-a82f-6e4378a9a571 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:26.926385 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655b01e-a7a6-47bd-a82f-6e4378a9a571 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:26.931378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4d814-cab4-4d08-87de-4b06be054024 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:26.937956 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f655b01e-a7a6-47bd-a82f-6e4378a9a571 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:26.946311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-f655b01e-a7a6-47bd-a82f-6e4378a9a571 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:26.946311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:26.946311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:26.946311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:26.946311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:26.946311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:26.946311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:26.946311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:26.946311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:26.946311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:26.946311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:26.946311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:26.946311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:26.946311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:26.946311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:26.946311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:26.947211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:26.947211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:26.947211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:26.947211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:26.947211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:26.947211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:26.947211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:26.947211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:26.947211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:26.947211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:26.947211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:26.947211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:26.947211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:26.947211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:26.947211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:26.947211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:26.947984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:26.947984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:26.947984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:26.947984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:26.947984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:26.947984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:26.947984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:26.947984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:26.947984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:26.947984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:26.947984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:26.947984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:26.950685 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 116/226] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:19:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:19:26.956847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655b01e-a7a6-47bd-a82f-6e4378a9a571 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:27.004560 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3112554-400e-48e4-bf22-68c90c7f3662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:27.005649 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3112554-400e-48e4-bf22-68c90c7f3662 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:27.006144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3112554-400e-48e4-bf22-68c90c7f3662 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:27.006640 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3112554-400e-48e4-bf22-68c90c7f3662 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:27.007581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3112554-400e-48e4-bf22-68c90c7f3662 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:27.008785 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3112554-400e-48e4-bf22-68c90c7f3662 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:27.010382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3112554-400e-48e4-bf22-68c90c7f3662 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:27.029803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 111/227] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:27.035937 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3112554-400e-48e4-bf22-68c90c7f3662 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:27.151725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655b01e-a7a6-47bd-a82f-6e4378a9a571 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:27.195023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:27.210971 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc49633-d344-434a-8239-c9685f42693e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:27.212165 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc49633-d344-434a-8239-c9685f42693e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:27.212458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc49633-d344-434a-8239-c9685f42693e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:27.212731 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc49633-d344-434a-8239-c9685f42693e None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:27.213516 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc49633-d344-434a-8239-c9685f42693e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:27.213827 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc49633-d344-434a-8239-c9685f42693e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:27.215221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc49633-d344-434a-8239-c9685f42693e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:27.226622 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 117/228] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:19:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:27.231980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc49633-d344-434a-8239-c9685f42693e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:27.238567 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3112554-400e-48e4-bf22-68c90c7f3662 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:27.290915 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c2ee9-e13e-46ad-91a5-ef5770273ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:27.291744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c2ee9-e13e-46ad-91a5-ef5770273ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:27.292026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c2ee9-e13e-46ad-91a5-ef5770273ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:27.292302 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c2ee9-e13e-46ad-91a5-ef5770273ba9 None admin] PATH_INFO: `/v3/users/5d1e906881c5465687c5968efcae625e` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:27.295748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6c2ee9-e13e-46ad-91a5-ef5770273ba9 None admin] RBAC: Authorizing `identity:get_user(user_id=5d1e906881c5465687c5968efcae625e)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:27.295748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6c2ee9-e13e-46ad-91a5-ef5770273ba9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:27.297153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 112/229] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:27 2022] GET /identity/v3/users/5d1e906881c5465687c5968efcae625e => generated 283 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:27.302440 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c2ee9-e13e-46ad-91a5-ef5770273ba9 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:27.319586 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:27.436234 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc49633-d344-434a-8239-c9685f42693e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:27.472862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c2ee9-e13e-46ad-91a5-ef5770273ba9 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:27.502142 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e69a7-f96b-4e9d-bc74-9385c09884a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:27.503019 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e69a7-f96b-4e9d-bc74-9385c09884a9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:27.503300 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e69a7-f96b-4e9d-bc74-9385c09884a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:27.503576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e69a7-f96b-4e9d-bc74-9385c09884a9 None admin] PATH_INFO: `/v3/users/5d1e906881c5465687c5968efcae625e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:27.507575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e69a7-f96b-4e9d-bc74-9385c09884a9 None admin] RBAC: Authorizing `identity:get_user(user_id=5d1e906881c5465687c5968efcae625e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:27.509047 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e69a7-f96b-4e9d-bc74-9385c09884a9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:27.510712 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 118/230] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:27 2022] GET /identity/v3/users/5d1e906881c5465687c5968efcae625e => generated 283 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:27.517042 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e69a7-f96b-4e9d-bc74-9385c09884a9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:27.561834 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5cb5c-85f2-4461-b6f9-29e9ea5fe0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:27.563283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5cb5c-85f2-4461-b6f9-29e9ea5fe0fd None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:27.563785 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5cb5c-85f2-4461-b6f9-29e9ea5fe0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:27.564278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5cb5c-85f2-4461-b6f9-29e9ea5fe0fd None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:27.567476 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5cb5c-85f2-4461-b6f9-29e9ea5fe0fd None admin] RBAC: Authorizing `identity:get_project(project_id=914fc5f0eaf745279782c417df7acf69)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:27.569232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5cb5c-85f2-4461-b6f9-29e9ea5fe0fd None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:27.571125 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 113/231] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:27 2022] GET /identity/v3/projects/914fc5f0eaf745279782c417df7acf69 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:27.576508 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5cb5c-85f2-4461-b6f9-29e9ea5fe0fd None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:27.585325 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:27.679054 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e69a7-f96b-4e9d-bc74-9385c09884a9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:27.727848 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cfabf-7039-46a1-8df2-8c3982289f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:27.728777 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cfabf-7039-46a1-8df2-8c3982289f39 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:27.729080 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cfabf-7039-46a1-8df2-8c3982289f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:27.729353 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cfabf-7039-46a1-8df2-8c3982289f39 None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:27.730633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70cfabf-7039-46a1-8df2-8c3982289f39 None admin] RBAC: Authorizing `identity:get_project(project_id=914fc5f0eaf745279782c417df7acf69)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:27.732068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70cfabf-7039-46a1-8df2-8c3982289f39 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:27.734366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 119/232] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:27 2022] GET /identity/v3/projects/914fc5f0eaf745279782c417df7acf69 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:27.736383 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:27.741995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cfabf-7039-46a1-8df2-8c3982289f39 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:27.773786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5cb5c-85f2-4461-b6f9-29e9ea5fe0fd None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:27.862690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc140040-468f-4ae9-bda0-891f8f72263f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:27.863521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc140040-468f-4ae9-bda0-891f8f72263f None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:27.863804 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc140040-468f-4ae9-bda0-891f8f72263f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:27.864083 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc140040-468f-4ae9-bda0-891f8f72263f None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69/users/5d1e906881c5465687c5968efcae625e/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:27.870390 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc140040-468f-4ae9-bda0-891f8f72263f None admin] RBAC: Authorizing `identity:create_grant(project_id=914fc5f0eaf745279782c417df7acf69, user_id=5d1e906881c5465687c5968efcae625e, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:27.871862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc140040-468f-4ae9-bda0-891f8f72263f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:27.879990 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cfabf-7039-46a1-8df2-8c3982289f39 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:27.905897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:27.916877 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e2ea0-9bc9-4839-83f3-cced0374f2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:27.917893 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e2ea0-9bc9-4839-83f3-cced0374f2dd None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:27.918173 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e2ea0-9bc9-4839-83f3-cced0374f2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:27.918444 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e2ea0-9bc9-4839-83f3-cced0374f2dd None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69/users/5d1e906881c5465687c5968efcae625e/roles/4552e69b96eb43f3ba6a9ca694d04ffb` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:27.920759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e2ea0-9bc9-4839-83f3-cced0374f2dd None admin] RBAC: Authorizing `identity:create_grant(project_id=914fc5f0eaf745279782c417df7acf69, user_id=5d1e906881c5465687c5968efcae625e, role_id=4552e69b96eb43f3ba6a9ca694d04ffb)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:27.922419 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e2ea0-9bc9-4839-83f3-cced0374f2dd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:27.992715 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 114/233] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:19:27 2022] PUT /identity/v3/projects/914fc5f0eaf745279782c417df7acf69/users/5d1e906881c5465687c5968efcae625e/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:19:28.024878 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42a823-081d-40b6-9a02-5af4a117aa1d None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:28.025212 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42a823-081d-40b6-9a02-5af4a117aa1d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:28.025539 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42a823-081d-40b6-9a02-5af4a117aa1d None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:28.027108 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 115/234] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:19:28.033408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b502f0-e12d-4bf8-bb5b-b899c0785b2a None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:28.033708 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b502f0-e12d-4bf8-bb5b-b899c0785b2a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:28.033969 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b502f0-e12d-4bf8-bb5b-b899c0785b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:28.054164 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-63b502f0-e12d-4bf8-bb5b-b899c0785b2a None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:28.055410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-63b502f0-e12d-4bf8-bb5b-b899c0785b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:28.056002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 120/235] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:19:27 2022] PUT /identity/v3/projects/914fc5f0eaf745279782c417df7acf69/users/5d1e906881c5465687c5968efcae625e/roles/4552e69b96eb43f3ba6a9ca694d04ffb => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:19:28.165806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 116/236] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:28.212712 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e2ea0-9bc9-4839-83f3-cced0374f2dd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:28.331439 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82841ea3-5975-44d4-a2e3-953957e23467 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:28.331946 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82841ea3-5975-44d4-a2e3-953957e23467 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:28.332019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82841ea3-5975-44d4-a2e3-953957e23467 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:28.333401 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 117/237] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:19:28.340543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8ed69-860c-4f86-a402-a7b2a689b352 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:28.340861 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8ed69-860c-4f86-a402-a7b2a689b352 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:28.341150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8ed69-860c-4f86-a402-a7b2a689b352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:28.348757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e2ea0-9bc9-4839-83f3-cced0374f2dd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:28.371684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-dbc8ed69-860c-4f86-a402-a7b2a689b352 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:28.372879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-dbc8ed69-860c-4f86-a402-a7b2a689b352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:28.385215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46665203-1c57-4dd9-9700-42024b791ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:28.386007 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46665203-1c57-4dd9-9700-42024b791ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:28.386283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46665203-1c57-4dd9-9700-42024b791ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:28.386557 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46665203-1c57-4dd9-9700-42024b791ba7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:28.398294 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46665203-1c57-4dd9-9700-42024b791ba7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:28.407010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-46665203-1c57-4dd9-9700-42024b791ba7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:19:28.407010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:28.407010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:28.407010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:28.407010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:28.407010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:28.407010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:28.407010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:28.407010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:28.407010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:28.407010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:28.407010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:28.407010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:28.407010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:28.407010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:28.407010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:28.409076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:28.409076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:28.409076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:28.409076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:28.409076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:28.409076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:28.409076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:28.409076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:28.409076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:28.409076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:28.409076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:28.409076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:28.409076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:28.409076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:28.409076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:28.409076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:28.411026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:28.411026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:28.411026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:28.411026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:28.411026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:28.411026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:28.411026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:28.411026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:28.411026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:28.411026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:28.411026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:19:28.411026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:28.411026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 121/238] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:28.417094 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46665203-1c57-4dd9-9700-42024b791ba7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:28.459840 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:28.475548 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:28.481695 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 118/239] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 07 08:19:28.516364 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8ed69-860c-4f86-a402-a7b2a689b352 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:28.544706 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46665203-1c57-4dd9-9700-42024b791ba7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:28.591116 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37aee03-de38-4b5f-85ff-2eae9e5ae87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:28.592350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37aee03-de38-4b5f-85ff-2eae9e5ae87d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:28.593703 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37aee03-de38-4b5f-85ff-2eae9e5ae87d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:28.594552 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37aee03-de38-4b5f-85ff-2eae9e5ae87d None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:28.596197 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37aee03-de38-4b5f-85ff-2eae9e5ae87d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:28.596795 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37aee03-de38-4b5f-85ff-2eae9e5ae87d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:28.599073 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37aee03-de38-4b5f-85ff-2eae9e5ae87d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:28.609872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 122/240] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:28.616126 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37aee03-de38-4b5f-85ff-2eae9e5ae87d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:28.666220 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8ed69-860c-4f86-a402-a7b2a689b352 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:28.703661 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07dc03-d4be-412c-971f-115c3a7272c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:28.704380 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07dc03-d4be-412c-971f-115c3a7272c6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:28.704846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07dc03-d4be-412c-971f-115c3a7272c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:28.705287 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07dc03-d4be-412c-971f-115c3a7272c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:28.706698 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07dc03-d4be-412c-971f-115c3a7272c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:28.708419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07dc03-d4be-412c-971f-115c3a7272c6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:28.710503 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 119/241] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:19:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:28.711860 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac284fca-1856-417e-91e8-779d00f0bfbd None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:28.712291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac284fca-1856-417e-91e8-779d00f0bfbd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:28.712707 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac284fca-1856-417e-91e8-779d00f0bfbd None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:28.714094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 120/242] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:28.716958 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac284fca-1856-417e-91e8-779d00f0bfbd None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:28.748102 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:28.752775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37aee03-de38-4b5f-85ff-2eae9e5ae87d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:28.791317 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f2bca-6c02-4786-8d79-ace0684f167f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:28.792490 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f2bca-6c02-4786-8d79-ace0684f167f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:28.793065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f2bca-6c02-4786-8d79-ace0684f167f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:28.793522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f2bca-6c02-4786-8d79-ace0684f167f None admin] PATH_INFO: `/v3/users/3ba4873935a64249b2ae9c8348646ce6` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:28.807044 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:28.812218 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f2bca-6c02-4786-8d79-ace0684f167f None admin] RBAC: Authorizing `identity:get_user(user_id=3ba4873935a64249b2ae9c8348646ce6)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:28.814016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f2bca-6c02-4786-8d79-ace0684f167f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:28.820586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 123/243] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:28 2022] GET /identity/v3/users/3ba4873935a64249b2ae9c8348646ce6 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:28.822083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646209a-9d91-46e2-aef4-64fdc818d97f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:28.824798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646209a-9d91-46e2-aef4-64fdc818d97f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:28.824798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646209a-9d91-46e2-aef4-64fdc818d97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:28.839584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9646209a-9d91-46e2-aef4-64fdc818d97f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:28.840985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9646209a-9d91-46e2-aef4-64fdc818d97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:28.847257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac284fca-1856-417e-91e8-779d00f0bfbd None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:28.891687 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663cc8e-7595-46a1-a667-dc80c0f62428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:28.892363 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663cc8e-7595-46a1-a667-dc80c0f62428 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:28.892633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663cc8e-7595-46a1-a667-dc80c0f62428 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:28.892904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663cc8e-7595-46a1-a667-dc80c0f62428 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:28.894344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663cc8e-7595-46a1-a667-dc80c0f62428 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:28.895819 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663cc8e-7595-46a1-a667-dc80c0f62428 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:28.925135 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 124/244] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:28.926619 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646209a-9d91-46e2-aef4-64fdc818d97f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:28.971078 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 121/245] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jun 7 08:19:28 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:19:28.977105 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663cc8e-7595-46a1-a667-dc80c0f62428 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:29.051728 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:29.066682 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646209a-9d91-46e2-aef4-64fdc818d97f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:29.107013 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a22b8e-0979-400e-a375-858fe1b555c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:29.107982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a22b8e-0979-400e-a375-858fe1b555c1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:29.109033 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a22b8e-0979-400e-a375-858fe1b555c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:29.109487 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a22b8e-0979-400e-a375-858fe1b555c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:29.188496 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a22b8e-0979-400e-a375-858fe1b555c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:29.190243 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a22b8e-0979-400e-a375-858fe1b555c1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:29.250362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663cc8e-7595-46a1-a667-dc80c0f62428 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:29.338602 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:29.342113 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:29.356242 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844ae97-b25b-4f20-be3c-724e9bedd5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:29.357391 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844ae97-b25b-4f20-be3c-724e9bedd5fa None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:29.357747 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 125/246] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:29.358148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844ae97-b25b-4f20-be3c-724e9bedd5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:29.359201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844ae97-b25b-4f20-be3c-724e9bedd5fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:29.363519 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a22b8e-0979-400e-a375-858fe1b555c1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:29.371450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844ae97-b25b-4f20-be3c-724e9bedd5fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:29.379491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-7844ae97-b25b-4f20-be3c-724e9bedd5fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:29.379491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:29.379491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:29.379491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:29.379491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:29.379491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:29.379491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:29.379491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:29.379491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:29.379491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:29.379491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:29.379491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:29.379491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:29.379491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:29.379491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:29.379491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:29.380412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:29.380412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:29.380412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:29.380412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:29.380412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:29.380412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:29.380412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:29.380412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:29.380412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:29.380412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:29.380412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:29.380412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:29.380412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:29.380412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:29.380412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:29.380412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:29.381201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:29.381201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:29.381201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:29.381201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:29.381201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:29.381201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:29.381201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:29.381201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:29.381201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:29.381201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:29.381201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:29.381201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:29.384009 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 122/247] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:29.391129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844ae97-b25b-4f20-be3c-724e9bedd5fa None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:29.478549 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a22b8e-0979-400e-a375-858fe1b555c1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:29.511052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844ae97-b25b-4f20-be3c-724e9bedd5fa None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:29.516685 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887028d-d353-4ef0-905d-6ae451392fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:29.517585 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887028d-d353-4ef0-905d-6ae451392fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:29.517867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887028d-d353-4ef0-905d-6ae451392fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:29.518141 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887028d-d353-4ef0-905d-6ae451392fb4 None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:29.519326 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887028d-d353-4ef0-905d-6ae451392fb4 None admin] RBAC: Authorizing `identity:get_project(project_id=914fc5f0eaf745279782c417df7acf69)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:29.520775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887028d-d353-4ef0-905d-6ae451392fb4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:29.522802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 126/248] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:29 2022] GET /identity/v3/projects/914fc5f0eaf745279782c417df7acf69 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:29.528745 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887028d-d353-4ef0-905d-6ae451392fb4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:29.554829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca5f83-d550-440f-87ac-bef5e9576672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:29.555911 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca5f83-d550-440f-87ac-bef5e9576672 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:29.556385 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca5f83-d550-440f-87ac-bef5e9576672 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:29.556903 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca5f83-d550-440f-87ac-bef5e9576672 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:29.558364 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca5f83-d550-440f-87ac-bef5e9576672 None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:29.558862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca5f83-d550-440f-87ac-bef5e9576672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:29.560428 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca5f83-d550-440f-87ac-bef5e9576672 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:29.570245 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 123/249] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:29.576138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca5f83-d550-440f-87ac-bef5e9576672 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:29.590446 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:29.632137 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:29.640098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887028d-d353-4ef0-905d-6ae451392fb4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:29.677957 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eddff24-2d21-477b-ad2b-95c18ce7ccc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:29.678625 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddff24-2d21-477b-ad2b-95c18ce7ccc4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:29.678908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddff24-2d21-477b-ad2b-95c18ce7ccc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:29.679186 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddff24-2d21-477b-ad2b-95c18ce7ccc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:29.680016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eddff24-2d21-477b-ad2b-95c18ce7ccc4 None admin] RBAC: Adding query filter params (role.id=5733a653e21248a7ab8ff7624b5f8ce9, scope.project.id=914fc5f0eaf745279782c417df7acf69, user.id=3ba4873935a64249b2ae9c8348646ce6) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:29.680331 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eddff24-2d21-477b-ad2b-95c18ce7ccc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:29.682231 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eddff24-2d21-477b-ad2b-95c18ce7ccc4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:29.688481 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 127/250] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:19:29 2022] GET /identity/v3/role_assignments?user.id=3ba4873935a64249b2ae9c8348646ce6&scope.project.id=914fc5f0eaf745279782c417df7acf69&role.id=5733a653e21248a7ab8ff7624b5f8ce9 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:29.693706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca5f83-d550-440f-87ac-bef5e9576672 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:29.733170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338dc72-abb9-40a7-b5c1-a43ec93ac3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:29.734140 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338dc72-abb9-40a7-b5c1-a43ec93ac3bc None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:29.734422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338dc72-abb9-40a7-b5c1-a43ec93ac3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:29.734695 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338dc72-abb9-40a7-b5c1-a43ec93ac3bc None admin] PATH_INFO: `/v3/users/d7d3784ac4294257add25142da9d8de4` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:29.735944 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338dc72-abb9-40a7-b5c1-a43ec93ac3bc None admin] RBAC: Authorizing `identity:get_user(user_id=d7d3784ac4294257add25142da9d8de4)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:29.737501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338dc72-abb9-40a7-b5c1-a43ec93ac3bc None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:29.739546 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 124/251] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:29 2022] GET /identity/v3/users/d7d3784ac4294257add25142da9d8de4 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:29.744827 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eddff24-2d21-477b-ad2b-95c18ce7ccc4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:29.896543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eddff24-2d21-477b-ad2b-95c18ce7ccc4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:29.924170 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:29.943688 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997fe2f-4913-450b-90da-cf6f340ffafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:29.944561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997fe2f-4913-450b-90da-cf6f340ffafd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:29.944847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997fe2f-4913-450b-90da-cf6f340ffafd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:29.945145 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997fe2f-4913-450b-90da-cf6f340ffafd None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:29.946347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997fe2f-4913-450b-90da-cf6f340ffafd None admin] RBAC: Authorizing `identity:get_project(project_id=914fc5f0eaf745279782c417df7acf69)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:29.948083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997fe2f-4913-450b-90da-cf6f340ffafd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:29.949820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 128/252] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:29 2022] GET /identity/v3/projects/914fc5f0eaf745279782c417df7acf69 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:29.955620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338dc72-abb9-40a7-b5c1-a43ec93ac3bc None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:30.107757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338dc72-abb9-40a7-b5c1-a43ec93ac3bc None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:30.125366 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:30.153692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3178a02-439b-40de-af2a-2fd69f090fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:30.153692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3178a02-439b-40de-af2a-2fd69f090fb5 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:30.153692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3178a02-439b-40de-af2a-2fd69f090fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:30.154019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3178a02-439b-40de-af2a-2fd69f090fb5 None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69/users/d7d3784ac4294257add25142da9d8de4/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:30.155880 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3178a02-439b-40de-af2a-2fd69f090fb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=914fc5f0eaf745279782c417df7acf69, user_id=d7d3784ac4294257add25142da9d8de4, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:30.157348 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3178a02-439b-40de-af2a-2fd69f090fb5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:30.214894 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 125/253] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:19:29 2022] PUT /identity/v3/projects/914fc5f0eaf745279782c417df7acf69/users/d7d3784ac4294257add25142da9d8de4/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:19:30.955045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788132a1-01a8-40f9-9628-3cdb010dd4f3 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:30.955597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788132a1-01a8-40f9-9628-3cdb010dd4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:30.955597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788132a1-01a8-40f9-9628-3cdb010dd4f3 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:30.956954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 129/254] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:30.962698 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b08c22-127d-4a8f-8c68-f7706edda23b None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:30.963140 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b08c22-127d-4a8f-8c68-f7706edda23b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:30.963339 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b08c22-127d-4a8f-8c68-f7706edda23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:30.990016 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-56b08c22-127d-4a8f-8c68-f7706edda23b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:30.991220 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-56b08c22-127d-4a8f-8c68-f7706edda23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:31.075304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 126/255] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:31.118772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788132a1-01a8-40f9-9628-3cdb010dd4f3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:31.238677 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788132a1-01a8-40f9-9628-3cdb010dd4f3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:31.273802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdc6f2-abfe-4c44-a760-165412f81910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:31.274603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdc6f2-abfe-4c44-a760-165412f81910 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:31.274886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdc6f2-abfe-4c44-a760-165412f81910 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:31.275160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdc6f2-abfe-4c44-a760-165412f81910 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:31.285722 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bdc6f2-abfe-4c44-a760-165412f81910 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:31.293524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-c7bdc6f2-abfe-4c44-a760-165412f81910 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:19:31.293524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:31.293524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:31.293524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:31.293524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:31.293524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:31.293524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:31.293524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:31.293524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:31.293524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:31.293524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:31.293524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:31.293524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:31.293524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:31.293524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:31.293524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:31.294393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:31.294393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:31.294393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:31.294393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:31.294393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:31.294393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:31.294393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:31.294393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:31.294393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:31.294393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:31.294393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:31.294393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:31.294393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:31.294393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:31.294393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:31.294393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:31.295154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:31.295154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:31.295154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:31.295154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:31.295154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:31.295154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:31.295154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:31.295154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:31.295154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:31.295154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:31.295154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:19:31.295154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:31.295154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 130/256] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:19:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:31.300932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b08c22-127d-4a8f-8c68-f7706edda23b None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:31.338429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02193c-86ff-4cee-aae3-5782537ac985 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:31.338731 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02193c-86ff-4cee-aae3-5782537ac985 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:31.338993 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02193c-86ff-4cee-aae3-5782537ac985 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:31.340364 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 131/257] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:19:31.346579 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a2fd0-ec46-4fe6-ab80-6c01a3656b24 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:31.346867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a2fd0-ec46-4fe6-ab80-6c01a3656b24 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:31.347151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a2fd0-ec46-4fe6-ab80-6c01a3656b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:31.368590 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a31a2fd0-ec46-4fe6-ab80-6c01a3656b24 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:31.369805 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a31a2fd0-ec46-4fe6-ab80-6c01a3656b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:31.418693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:31.439161 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 132/258] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:31.473657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a2fd0-ec46-4fe6-ab80-6c01a3656b24 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:31.493155 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b08c22-127d-4a8f-8c68-f7706edda23b None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:31.494936 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:31.553831 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f435e-bd22-42f1-b02d-0b642dec9218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:31.554952 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f435e-bd22-42f1-b02d-0b642dec9218 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:31.555373 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f435e-bd22-42f1-b02d-0b642dec9218 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:31.555795 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f435e-bd22-42f1-b02d-0b642dec9218 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:31.556677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f435e-bd22-42f1-b02d-0b642dec9218 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:31.557149 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f435e-bd22-42f1-b02d-0b642dec9218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:31.558671 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f435e-bd22-42f1-b02d-0b642dec9218 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:31.572629 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 127/259] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:31.578919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f435e-bd22-42f1-b02d-0b642dec9218 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:31.724665 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a2fd0-ec46-4fe6-ab80-6c01a3656b24 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:31.772655 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f435e-bd22-42f1-b02d-0b642dec9218 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:31.813716 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41921fd7-26a3-4abd-9cea-dd654021e078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:31.814539 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41921fd7-26a3-4abd-9cea-dd654021e078 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:31.814819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41921fd7-26a3-4abd-9cea-dd654021e078 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:31.815090 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41921fd7-26a3-4abd-9cea-dd654021e078 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:31.831641 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41921fd7-26a3-4abd-9cea-dd654021e078 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:31.839094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105dee25-f3a0-4876-a0ed-5a1a6e785da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:31.839972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dee25-f3a0-4876-a0ed-5a1a6e785da8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:31.840152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-41921fd7-26a3-4abd-9cea-dd654021e078 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:31.840152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:31.840152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:31.840152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:31.840152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:31.840152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:31.840152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:31.840152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:31.840152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:31.840152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:31.840152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:31.840152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:31.840152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:31.840152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:31.840152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:31.840152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:31.840984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:31.840984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:31.840984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:31.840984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:31.840984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:31.840984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:31.840984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:31.840984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:31.840984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:31.840984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:31.840984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:31.840984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:31.840984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:31.840984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:31.840984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:31.840984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:31.841787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:31.841787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:31.841787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:31.841787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:31.841787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:31.841787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:31.841787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:31.841787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:31.841787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:31.841787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:31.841787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:31.841787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:31.842399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dee25-f3a0-4876-a0ed-5a1a6e785da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:31.842399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dee25-f3a0-4876-a0ed-5a1a6e785da8 None admin] PATH_INFO: `/v3/users/5d1e906881c5465687c5968efcae625e` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:31.845716 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105dee25-f3a0-4876-a0ed-5a1a6e785da8 None admin] RBAC: Authorizing `identity:get_user(user_id=5d1e906881c5465687c5968efcae625e)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:31.847181 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105dee25-f3a0-4876-a0ed-5a1a6e785da8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:31.849288 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 128/261] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:31 2022] GET /identity/v3/users/5d1e906881c5465687c5968efcae625e => generated 283 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:31.849401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 133/261] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:19:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:31.850058 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105dee25-f3a0-4876-a0ed-5a1a6e785da8 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:31.854427 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41921fd7-26a3-4abd-9cea-dd654021e078 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:31.918245 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:31.940155 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:32.065080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105dee25-f3a0-4876-a0ed-5a1a6e785da8 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:32.084866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41921fd7-26a3-4abd-9cea-dd654021e078 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:32.137642 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160edb1-d328-4ff1-b191-99d4099fc906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:32.138666 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160edb1-d328-4ff1-b191-99d4099fc906 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:32.139145 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160edb1-d328-4ff1-b191-99d4099fc906 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:32.139658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160edb1-d328-4ff1-b191-99d4099fc906 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:32.181718 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a16ed-c903-4243-bcb9-4f6663d04cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:32.182403 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a16ed-c903-4243-bcb9-4f6663d04cce None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:32.182681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a16ed-c903-4243-bcb9-4f6663d04cce None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:32.182950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a16ed-c903-4243-bcb9-4f6663d04cce None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:32.183706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a16ed-c903-4243-bcb9-4f6663d04cce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:32.184010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a16ed-c903-4243-bcb9-4f6663d04cce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:32.185412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a16ed-c903-4243-bcb9-4f6663d04cce None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:32.211197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 129/262] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:19:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:32.216276 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a16ed-c903-4243-bcb9-4f6663d04cce None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:32.218718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5160edb1-d328-4ff1-b191-99d4099fc906 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:32.222266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5160edb1-d328-4ff1-b191-99d4099fc906 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:32.264260 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:32.335869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 134/263] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 08:19:32.343039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160edb1-d328-4ff1-b191-99d4099fc906 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:32.469655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160edb1-d328-4ff1-b191-99d4099fc906 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:32.483524 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:32.503172 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a16ed-c903-4243-bcb9-4f6663d04cce None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:32.511589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913a540-3787-475e-a0d8-f59a800afdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:32.512518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913a540-3787-475e-a0d8-f59a800afdf3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:32.512795 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913a540-3787-475e-a0d8-f59a800afdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:32.513109 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913a540-3787-475e-a0d8-f59a800afdf3 None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:32.514827 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913a540-3787-475e-a0d8-f59a800afdf3 None admin] RBAC: Authorizing `identity:get_project(project_id=914fc5f0eaf745279782c417df7acf69)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:32.516456 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913a540-3787-475e-a0d8-f59a800afdf3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:32.518250 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 135/264] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:32 2022] GET /identity/v3/projects/914fc5f0eaf745279782c417df7acf69 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:32.519445 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf6009-21ca-42c9-8b21-63876b89e54e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:32.519726 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf6009-21ca-42c9-8b21-63876b89e54e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:32.520000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf6009-21ca-42c9-8b21-63876b89e54e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:32.521273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 136/265] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:32.525867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf6009-21ca-42c9-8b21-63876b89e54e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:32.536313 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:32.557890 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204a9e7-d6e6-4bbc-b479-d77e76ef37ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:32.558780 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204a9e7-d6e6-4bbc-b479-d77e76ef37ac None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:32.559056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204a9e7-d6e6-4bbc-b479-d77e76ef37ac None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:32.559327 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204a9e7-d6e6-4bbc-b479-d77e76ef37ac None admin] PATH_INFO: `/v3/users/5d1e906881c5465687c5968efcae625e` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:32.561979 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204a9e7-d6e6-4bbc-b479-d77e76ef37ac None admin] RBAC: Authorizing `identity:get_user(user_id=5d1e906881c5465687c5968efcae625e)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:32.563439 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204a9e7-d6e6-4bbc-b479-d77e76ef37ac None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:32.565081 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 130/266] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:32 2022] GET /identity/v3/users/5d1e906881c5465687c5968efcae625e => generated 283 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:32.566222 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b7d3c-cc7f-4578-b4ef-f9327c667a5b None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:32.566502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b7d3c-cc7f-4578-b4ef-f9327c667a5b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:32.566777 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b7d3c-cc7f-4578-b4ef-f9327c667a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:32.610499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-022b7d3c-cc7f-4578-b4ef-f9327c667a5b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:32.611919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-022b7d3c-cc7f-4578-b4ef-f9327c667a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:32.661486 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf6009-21ca-42c9-8b21-63876b89e54e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:32.731676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01022c55-8cd1-49b9-a239-9c4d03857abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:32.732582 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01022c55-8cd1-49b9-a239-9c4d03857abe None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:32.734083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01022c55-8cd1-49b9-a239-9c4d03857abe None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:32.734534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01022c55-8cd1-49b9-a239-9c4d03857abe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:32.736914 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01022c55-8cd1-49b9-a239-9c4d03857abe None admin] RBAC: Adding query filter params (role.id=8c63c00418a346b29f7db3f87a404e70, scope.project.id=914fc5f0eaf745279782c417df7acf69, user.id=5d1e906881c5465687c5968efcae625e) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:32.737399 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01022c55-8cd1-49b9-a239-9c4d03857abe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:32.739328 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01022c55-8cd1-49b9-a239-9c4d03857abe None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:32.743010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 131/267] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:32.744032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b7d3c-cc7f-4578-b4ef-f9327c667a5b None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:32.757739 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 137/268] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:19:32 2022] GET /identity/v3/role_assignments?user.id=5d1e906881c5465687c5968efcae625e&scope.project.id=914fc5f0eaf745279782c417df7acf69&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:32.783576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01022c55-8cd1-49b9-a239-9c4d03857abe None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:32.880028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:32.892851 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b7d3c-cc7f-4578-b4ef-f9327c667a5b None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:32.942764 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:32.943119 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5058cbc-a794-41e6-ad25-2cfcbe2207e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:32.943789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5058cbc-a794-41e6-ad25-2cfcbe2207e5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:32.944118 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5058cbc-a794-41e6-ad25-2cfcbe2207e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:32.944548 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5058cbc-a794-41e6-ad25-2cfcbe2207e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:32.982829 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01022c55-8cd1-49b9-a239-9c4d03857abe None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:33.034382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10502736-9a05-455a-a401-107c47d9e29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:33.035437 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10502736-9a05-455a-a401-107c47d9e29c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:33.035867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10502736-9a05-455a-a401-107c47d9e29c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:33.036283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10502736-9a05-455a-a401-107c47d9e29c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:33.041710 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10502736-9a05-455a-a401-107c47d9e29c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:33.043393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10502736-9a05-455a-a401-107c47d9e29c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:33.045279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 138/269] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:19:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:33.051049 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10502736-9a05-455a-a401-107c47d9e29c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:33.064767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5058cbc-a794-41e6-ad25-2cfcbe2207e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:33.066377 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5058cbc-a794-41e6-ad25-2cfcbe2207e5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:33.169118 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:33.186032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 132/270] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:33.193886 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5058cbc-a794-41e6-ad25-2cfcbe2207e5 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:33.236634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10502736-9a05-455a-a401-107c47d9e29c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:33.287590 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c14cc-2846-47ac-a13f-fedb991b5d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:33.288645 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c14cc-2846-47ac-a13f-fedb991b5d51 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:33.288923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c14cc-2846-47ac-a13f-fedb991b5d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:33.289230 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c14cc-2846-47ac-a13f-fedb991b5d51 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:33.289942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c14cc-2846-47ac-a13f-fedb991b5d51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:33.291573 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c14cc-2846-47ac-a13f-fedb991b5d51 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:33.301148 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:33.325200 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5058cbc-a794-41e6-ad25-2cfcbe2207e5 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:33.359801 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375f032-801d-41ab-ae56-166588b39ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:33.360667 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375f032-801d-41ab-ae56-166588b39ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:33.360946 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375f032-801d-41ab-ae56-166588b39ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:33.361243 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375f032-801d-41ab-ae56-166588b39ad4 None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:33.362484 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375f032-801d-41ab-ae56-166588b39ad4 None admin] RBAC: Authorizing `identity:get_project(project_id=914fc5f0eaf745279782c417df7acf69)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:33.363910 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375f032-801d-41ab-ae56-166588b39ad4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:33.365605 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 133/271] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:33 2022] GET /identity/v3/projects/914fc5f0eaf745279782c417df7acf69 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:33.370663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087082f-ff80-4cf2-b986-beba624a9fd4 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:33.370663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087082f-ff80-4cf2-b986-beba624a9fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:33.370663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087082f-ff80-4cf2-b986-beba624a9fd4 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:33.370663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 134/272] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:33.370663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c2868-3879-4c2b-88ac-7aa54b5f571a None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:33.370663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c2868-3879-4c2b-88ac-7aa54b5f571a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:33.371065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c2868-3879-4c2b-88ac-7aa54b5f571a None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:33.371750 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 135/273] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:33.372455 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c2868-3879-4c2b-88ac-7aa54b5f571a None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:33.385561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 139/274] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jun 7 08:19:33 2022] POST /identity/v3/users => generated 293 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:19:33.386826 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d1449-795f-42e3-927c-e3656dd448a6 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:33.391190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d1449-795f-42e3-927c-e3656dd448a6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:33.391190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d1449-795f-42e3-927c-e3656dd448a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:33.407943 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b96d1449-795f-42e3-927c-e3656dd448a6 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:33.410151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b96d1449-795f-42e3-927c-e3656dd448a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:33.447517 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:33.497410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c2868-3879-4c2b-88ac-7aa54b5f571a None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:33.513670 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 140/275] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:33.515750 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a474b71-0554-4184-9284-661d5de93d79 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:33.516039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a474b71-0554-4184-9284-661d5de93d79 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:33.516314 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a474b71-0554-4184-9284-661d5de93d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:33.535997 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09435437-722e-4fa4-8e16-476b4ba2ebf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:33.536818 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09435437-722e-4fa4-8e16-476b4ba2ebf9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:33.537217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09435437-722e-4fa4-8e16-476b4ba2ebf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:33.537589 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09435437-722e-4fa4-8e16-476b4ba2ebf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:33.538535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09435437-722e-4fa4-8e16-476b4ba2ebf9 None admin] RBAC: Adding query filter params (role.id=4552e69b96eb43f3ba6a9ca694d04ffb, scope.project.id=914fc5f0eaf745279782c417df7acf69, user.id=5d1e906881c5465687c5968efcae625e) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:33.538944 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09435437-722e-4fa4-8e16-476b4ba2ebf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:33.540521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09435437-722e-4fa4-8e16-476b4ba2ebf9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:33.541237 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8a474b71-0554-4184-9284-661d5de93d79 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:33.542606 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8a474b71-0554-4184-9284-661d5de93d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:33.548279 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 136/276] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:19:33 2022] GET /identity/v3/role_assignments?user.id=5d1e906881c5465687c5968efcae625e&scope.project.id=914fc5f0eaf745279782c417df7acf69&role.id=4552e69b96eb43f3ba6a9ca694d04ffb => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:33.550169 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077113f-e8db-44d9-87d8-5f5022c7cf73 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:33.550451 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077113f-e8db-44d9-87d8-5f5022c7cf73 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:33.550724 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077113f-e8db-44d9-87d8-5f5022c7cf73 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:33.552056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 137/277] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:33.556838 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077113f-e8db-44d9-87d8-5f5022c7cf73 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:33.650546 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 141/278] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:33.651961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e727a4-39dd-43e9-ad18-26dac71742bb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:33.652252 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e727a4-39dd-43e9-ad18-26dac71742bb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:33.652537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e727a4-39dd-43e9-ad18-26dac71742bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:33.678970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-03e727a4-39dd-43e9-ad18-26dac71742bb None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:33.680182 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-03e727a4-39dd-43e9-ad18-26dac71742bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:33.687136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077113f-e8db-44d9-87d8-5f5022c7cf73 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:33.722500 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:33.735304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8529c8a-f147-40eb-b399-ab94d8f49442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:33.737082 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8529c8a-f147-40eb-b399-ab94d8f49442 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:33.737369 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8529c8a-f147-40eb-b399-ab94d8f49442 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:33.740681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8529c8a-f147-40eb-b399-ab94d8f49442 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:33.752501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8529c8a-f147-40eb-b399-ab94d8f49442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:33.756919 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:33.763126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-c8529c8a-f147-40eb-b399-ab94d8f49442 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:33.763126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:33.763126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:33.763126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:33.763126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:33.763126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:33.763126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:33.763126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:33.763126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:33.763126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:33.763126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:33.763126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:33.763126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:33.763126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:33.763126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:33.763126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:33.764052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:33.764052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:33.764052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:33.764052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:33.764052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:33.764052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:33.764052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:33.764052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:33.764052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:33.764052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:33.764052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:33.764052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:33.764052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:33.764052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:33.764052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:33.764052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:33.764891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:33.764891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:33.764891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:33.764891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:33.764891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:33.764891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:33.764891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:33.764891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:33.764891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:33.764891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:33.764891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:33.764891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:33.764891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 138/279] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:33.765649 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8529c8a-f147-40eb-b399-ab94d8f49442 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:33.778651 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 142/280] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:33.779519 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e727a4-39dd-43e9-ad18-26dac71742bb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:33.899039 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8529c8a-f147-40eb-b399-ab94d8f49442 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:33.903938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e727a4-39dd-43e9-ad18-26dac71742bb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:33.942920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae00cc6-7402-41a2-9784-70c4e91de38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:33.944171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae00cc6-7402-41a2-9784-70c4e91de38e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:33.944583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae00cc6-7402-41a2-9784-70c4e91de38e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:33.944865 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae00cc6-7402-41a2-9784-70c4e91de38e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:33.948242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d8fe8-a00d-44a4-8426-50bc2c919f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:33.949051 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d8fe8-a00d-44a4-8426-50bc2c919f2b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:33.949444 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d8fe8-a00d-44a4-8426-50bc2c919f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:33.949818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d8fe8-a00d-44a4-8426-50bc2c919f2b None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:33.950686 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d8fe8-a00d-44a4-8426-50bc2c919f2b None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:33.951094 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d8fe8-a00d-44a4-8426-50bc2c919f2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:33.953130 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d8fe8-a00d-44a4-8426-50bc2c919f2b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:33.957284 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae00cc6-7402-41a2-9784-70c4e91de38e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:33.962813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 143/281] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:33.963774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d8fe8-a00d-44a4-8426-50bc2c919f2b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:33.968881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-7ae00cc6-7402-41a2-9784-70c4e91de38e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:19:33.968881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:33.968881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:33.968881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:33.968881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:33.968881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:33.968881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:33.968881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:33.968881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:33.968881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:33.968881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:33.968881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:33.968881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:33.968881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:33.968881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:33.968881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:33.969989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:33.969989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:33.969989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:33.969989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:33.969989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:33.969989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:33.969989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:33.969989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:33.969989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:33.969989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:33.969989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:33.969989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:33.969989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:33.969989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:33.969989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:33.969989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:33.970972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:33.970972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:33.970972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:33.970972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:33.970972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:33.970972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:33.970972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:33.970972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:33.970972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:33.970972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:33.970972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:19:33.970972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:33.972687 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 139/282] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:33.973487 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae00cc6-7402-41a2-9784-70c4e91de38e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:34.029633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:34.033146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:34.089750 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae00cc6-7402-41a2-9784-70c4e91de38e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:34.093217 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d8fe8-a00d-44a4-8426-50bc2c919f2b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:34.128547 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76052df0-0dcb-4f3a-bc01-ac9d12f547ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:34.129567 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad859d7-73a5-43ad-b456-b59c942742d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:34.129967 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76052df0-0dcb-4f3a-bc01-ac9d12f547ee None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:34.130424 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76052df0-0dcb-4f3a-bc01-ac9d12f547ee None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:34.130645 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad859d7-73a5-43ad-b456-b59c942742d1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:34.130994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76052df0-0dcb-4f3a-bc01-ac9d12f547ee None admin] PATH_INFO: `/v3/users/7b2d0d20f4f3479e8c7fbaee2f25560e` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:34.131104 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad859d7-73a5-43ad-b456-b59c942742d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:34.131465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad859d7-73a5-43ad-b456-b59c942742d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:34.143451 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad859d7-73a5-43ad-b456-b59c942742d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:34.147088 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76052df0-0dcb-4f3a-bc01-ac9d12f547ee None admin] RBAC: Authorizing `identity:get_user(user_id=7b2d0d20f4f3479e8c7fbaee2f25560e)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:34.148751 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76052df0-0dcb-4f3a-bc01-ac9d12f547ee None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:34.150649 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 140/283] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:33 2022] GET /identity/v3/users/7b2d0d20f4f3479e8c7fbaee2f25560e => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:34.151629 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76052df0-0dcb-4f3a-bc01-ac9d12f547ee None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:34.157684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-3ad859d7-73a5-43ad-b456-b59c942742d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:34.157684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:34.157684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:34.157684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:34.157684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:34.157684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:34.157684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:34.157684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:34.157684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:34.157684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:34.157684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:34.157684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:34.157684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:34.157684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:34.157684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:34.157684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:34.158575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:34.158575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:34.158575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:34.158575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:34.158575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:34.158575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:34.158575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:34.158575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:34.158575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:34.158575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:34.158575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:34.158575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:34.158575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:34.158575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:34.158575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:34.158575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:34.159349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:34.159349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:34.159349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:34.159349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:34.159349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:34.159349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:34.159349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:34.159349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:34.159349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:34.159349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:34.159349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:34.159349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:34.159349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 144/284] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:34.160457 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad859d7-73a5-43ad-b456-b59c942742d1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:34.278052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76052df0-0dcb-4f3a-bc01-ac9d12f547ee None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:34.280784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:34.283332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad859d7-73a5-43ad-b456-b59c942742d1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:34.319842 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:34.323201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147951e1-8081-49f8-af19-814227798796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:34.324215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147951e1-8081-49f8-af19-814227798796 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:34.324679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147951e1-8081-49f8-af19-814227798796 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:34.325231 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147951e1-8081-49f8-af19-814227798796 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:34.326153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147951e1-8081-49f8-af19-814227798796 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:34.326463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147951e1-8081-49f8-af19-814227798796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:34.328426 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147951e1-8081-49f8-af19-814227798796 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:34.341021 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61b4be-d464-4865-97c3-e3d1b5cd5615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:34.341021 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61b4be-d464-4865-97c3-e3d1b5cd5615 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:34.341021 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61b4be-d464-4865-97c3-e3d1b5cd5615 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:34.341021 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61b4be-d464-4865-97c3-e3d1b5cd5615 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:34.345736 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61b4be-d464-4865-97c3-e3d1b5cd5615 None admin] RBAC: Adding query filter params (role.id=67111011e94c4e08928dd5e6b7a46af1, user.id=7b2d0d20f4f3479e8c7fbaee2f25560e, scope.system=all) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:34.345736 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61b4be-d464-4865-97c3-e3d1b5cd5615 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:34.345736 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61b4be-d464-4865-97c3-e3d1b5cd5615 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:34.345959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 141/285] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:34.345959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147951e1-8081-49f8-af19-814227798796 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:34.346097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 145/286] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Jun 7 08:19:34 2022] GET /identity/v3/role_assignments?user.id=7b2d0d20f4f3479e8c7fbaee2f25560e&scope.system=all&role.id=67111011e94c4e08928dd5e6b7a46af1 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:34.353265 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61b4be-d464-4865-97c3-e3d1b5cd5615 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:34.456912 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147951e1-8081-49f8-af19-814227798796 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:34.505729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1283e-f016-4fca-a81c-4caf27b04f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:34.506422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1283e-f016-4fca-a81c-4caf27b04f17 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:34.506709 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1283e-f016-4fca-a81c-4caf27b04f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:34.506984 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1283e-f016-4fca-a81c-4caf27b04f17 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:34.507754 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1283e-f016-4fca-a81c-4caf27b04f17 None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:34.508066 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1283e-f016-4fca-a81c-4caf27b04f17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:34.509522 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1283e-f016-4fca-a81c-4caf27b04f17 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:34.528261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 142/287] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:34.533959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1283e-f016-4fca-a81c-4caf27b04f17 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:34.572596 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61b4be-d464-4865-97c3-e3d1b5cd5615 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:34.587897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:34.614641 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690924e-83a4-4c15-ae53-fd0cb38d031e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:34.615526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690924e-83a4-4c15-ae53-fd0cb38d031e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:34.615811 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690924e-83a4-4c15-ae53-fd0cb38d031e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:34.616092 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690924e-83a4-4c15-ae53-fd0cb38d031e None admin] PATH_INFO: `/v3/users/3ba4873935a64249b2ae9c8348646ce6` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:34.617326 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690924e-83a4-4c15-ae53-fd0cb38d031e None admin] RBAC: Authorizing `identity:get_user(user_id=3ba4873935a64249b2ae9c8348646ce6)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:34.618811 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690924e-83a4-4c15-ae53-fd0cb38d031e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:34.622042 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 146/288] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:34 2022] GET /identity/v3/users/3ba4873935a64249b2ae9c8348646ce6 => generated 297 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:34.628025 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690924e-83a4-4c15-ae53-fd0cb38d031e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:34.688024 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1283e-f016-4fca-a81c-4caf27b04f17 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:34.711982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:34.726554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d7a31-da53-4dd2-aa7a-14855cae79a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:34.727978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d7a31-da53-4dd2-aa7a-14855cae79a2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:34.728467 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d7a31-da53-4dd2-aa7a-14855cae79a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:34.728943 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d7a31-da53-4dd2-aa7a-14855cae79a2 None admin] PATH_INFO: `/v3/users/d7d3784ac4294257add25142da9d8de4` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:34.730377 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d7a31-da53-4dd2-aa7a-14855cae79a2 None admin] RBAC: Authorizing `identity:get_user(user_id=d7d3784ac4294257add25142da9d8de4)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:34.732054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d7a31-da53-4dd2-aa7a-14855cae79a2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:34.738647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 143/289] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:34 2022] GET /identity/v3/users/d7d3784ac4294257add25142da9d8de4 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:34.740773 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d7a31-da53-4dd2-aa7a-14855cae79a2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:34.856522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690924e-83a4-4c15-ae53-fd0cb38d031e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:34.885370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:34.896102 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc26de-f93c-4c55-b376-a3943818cec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:34.897286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc26de-f93c-4c55-b376-a3943818cec2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:34.897772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc26de-f93c-4c55-b376-a3943818cec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:34.898247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc26de-f93c-4c55-b376-a3943818cec2 None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:34.899642 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbc26de-f93c-4c55-b376-a3943818cec2 None admin] RBAC: Authorizing `identity:get_project(project_id=914fc5f0eaf745279782c417df7acf69)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:34.900580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d7a31-da53-4dd2-aa7a-14855cae79a2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:34.908382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbc26de-f93c-4c55-b376-a3943818cec2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:34.908382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 147/290] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:34 2022] GET /identity/v3/projects/914fc5f0eaf745279782c417df7acf69 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:34.908830 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc26de-f93c-4c55-b376-a3943818cec2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:34.976882 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfdb0b-e58b-42e5-a1db-3af9a5d0f9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:34.977575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfdb0b-e58b-42e5-a1db-3af9a5d0f9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:34.977863 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfdb0b-e58b-42e5-a1db-3af9a5d0f9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:34.978400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfdb0b-e58b-42e5-a1db-3af9a5d0f9c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:35.094662 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfdb0b-e58b-42e5-a1db-3af9a5d0f9c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:35.096149 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfdb0b-e58b-42e5-a1db-3af9a5d0f9c9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:35.104836 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc26de-f93c-4c55-b376-a3943818cec2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:35.113171 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:35.148613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5d36b-69c2-454e-9ede-d29e025e3145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:35.149797 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5d36b-69c2-454e-9ede-d29e025e3145 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:35.150233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5d36b-69c2-454e-9ede-d29e025e3145 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:35.150661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5d36b-69c2-454e-9ede-d29e025e3145 None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69/users/3ba4873935a64249b2ae9c8348646ce6/roles/5733a653e21248a7ab8ff7624b5f8ce9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:35.154745 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5d36b-69c2-454e-9ede-d29e025e3145 None admin] RBAC: Authorizing `identity:create_grant(project_id=914fc5f0eaf745279782c417df7acf69, user_id=3ba4873935a64249b2ae9c8348646ce6, role_id=5733a653e21248a7ab8ff7624b5f8ce9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:35.157544 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5d36b-69c2-454e-9ede-d29e025e3145 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:35.231682 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 148/291] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:19:34 2022] PUT /identity/v3/projects/914fc5f0eaf745279782c417df7acf69/users/3ba4873935a64249b2ae9c8348646ce6/roles/5733a653e21248a7ab8ff7624b5f8ce9 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:19:35.243102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:35.256730 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 144/292] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:35.262531 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5d36b-69c2-454e-9ede-d29e025e3145 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:35.400943 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5d36b-69c2-454e-9ede-d29e025e3145 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:35.450121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30584b-3847-4f57-8207-8cd0b07a753f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:35.451009 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30584b-3847-4f57-8207-8cd0b07a753f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:35.451295 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30584b-3847-4f57-8207-8cd0b07a753f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:35.451574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30584b-3847-4f57-8207-8cd0b07a753f None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:35.453723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30584b-3847-4f57-8207-8cd0b07a753f None admin] RBAC: Authorizing `identity:get_project(project_id=914fc5f0eaf745279782c417df7acf69)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:35.456074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30584b-3847-4f57-8207-8cd0b07a753f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:35.463314 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 149/293] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:35 2022] GET /identity/v3/projects/914fc5f0eaf745279782c417df7acf69 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:35.464746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfdb0b-e58b-42e5-a1db-3af9a5d0f9c9 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:35.578409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfdb0b-e58b-42e5-a1db-3af9a5d0f9c9 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:35.616400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a54658-1566-4470-8875-e5ce4a63e0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:35.617221 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a54658-1566-4470-8875-e5ce4a63e0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:35.617509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a54658-1566-4470-8875-e5ce4a63e0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:35.617791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a54658-1566-4470-8875-e5ce4a63e0d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:35.618638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a54658-1566-4470-8875-e5ce4a63e0d9 None admin] RBAC: Adding query filter params (role.id=67111011e94c4e08928dd5e6b7a46af1, scope.project.id=914fc5f0eaf745279782c417df7acf69, user.id=d7d3784ac4294257add25142da9d8de4) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:35.618955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a54658-1566-4470-8875-e5ce4a63e0d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:35.620403 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a54658-1566-4470-8875-e5ce4a63e0d9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:35.627106 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 145/294] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:19:35 2022] GET /identity/v3/role_assignments?user.id=d7d3784ac4294257add25142da9d8de4&scope.project.id=914fc5f0eaf745279782c417df7acf69&role.id=67111011e94c4e08928dd5e6b7a46af1 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:36.187579 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44523d56-39ef-448e-acf9-682b5c65edf3 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:36.187875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44523d56-39ef-448e-acf9-682b5c65edf3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:36.188158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44523d56-39ef-448e-acf9-682b5c65edf3 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:36.189567 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 150/295] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:36.195620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e4ed8-fede-42fe-bfff-c189bf09115c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:36.196058 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e4ed8-fede-42fe-bfff-c189bf09115c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:36.196253 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e4ed8-fede-42fe-bfff-c189bf09115c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:36.215737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-435e4ed8-fede-42fe-bfff-c189bf09115c None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:36.216943 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-435e4ed8-fede-42fe-bfff-c189bf09115c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:36.323927 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:36.339306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 146/296] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:36.363076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44523d56-39ef-448e-acf9-682b5c65edf3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:36.384827 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d973c5-eb11-4065-b82a-65a6615fd75e None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:36.385158 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d973c5-eb11-4065-b82a-65a6615fd75e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:36.385412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d973c5-eb11-4065-b82a-65a6615fd75e None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:36.387380 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 147/297] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:19:36.394291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bacbef-7807-44df-b01d-e59079326d78 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:36.394583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bacbef-7807-44df-b01d-e59079326d78 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:36.394870 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bacbef-7807-44df-b01d-e59079326d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:36.404424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:36.416710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-22bacbef-7807-44df-b01d-e59079326d78 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:36.418384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-22bacbef-7807-44df-b01d-e59079326d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:36.481122 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44523d56-39ef-448e-acf9-682b5c65edf3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:36.513095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 148/298] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:36.523324 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9796845-d36b-4ae6-863e-30db12c337b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:36.524454 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9796845-d36b-4ae6-863e-30db12c337b2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:36.524964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9796845-d36b-4ae6-863e-30db12c337b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:36.525504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9796845-d36b-4ae6-863e-30db12c337b2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:36.536903 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bacbef-7807-44df-b01d-e59079326d78 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:36.537264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9796845-d36b-4ae6-863e-30db12c337b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:36.545398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a9796845-d36b-4ae6-863e-30db12c337b2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:19:36.545398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:36.545398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:36.545398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:36.545398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:36.545398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:36.545398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:36.545398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:36.545398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:36.545398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:36.545398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:36.545398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:36.545398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:36.545398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:36.545398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:36.545398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:36.546321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:36.546321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:36.546321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:36.546321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:36.546321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:36.546321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:36.546321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:36.546321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:36.546321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:36.546321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:36.546321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:36.546321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:36.546321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:36.546321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:36.546321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:36.546321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:36.547098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:36.547098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:36.547098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:36.547098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:36.547098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:36.547098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:36.547098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:36.547098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:36.547098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:36.547098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:36.547098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:19:36.547098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:36.549815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 151/299] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:36.556088 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9796845-d36b-4ae6-863e-30db12c337b2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:36.662479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bacbef-7807-44df-b01d-e59079326d78 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:36.680552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:36.684057 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9796845-d36b-4ae6-863e-30db12c337b2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:36.688157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:36.699332 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d211ab-cea9-42c4-a95e-00c03c65677c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:36.700408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d211ab-cea9-42c4-a95e-00c03c65677c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:36.700892 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d211ab-cea9-42c4-a95e-00c03c65677c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:36.701393 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d211ab-cea9-42c4-a95e-00c03c65677c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:36.703417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d211ab-cea9-42c4-a95e-00c03c65677c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:36.705175 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d211ab-cea9-42c4-a95e-00c03c65677c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:36.707281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 149/300] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:19:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:36.712597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d211ab-cea9-42c4-a95e-00c03c65677c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:36.730591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f4b5c-3dd7-4f34-91f4-e9f222b9f052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:36.737224 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f4b5c-3dd7-4f34-91f4-e9f222b9f052 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:36.737224 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f4b5c-3dd7-4f34-91f4-e9f222b9f052 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:36.737224 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f4b5c-3dd7-4f34-91f4-e9f222b9f052 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:36.737224 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f4b5c-3dd7-4f34-91f4-e9f222b9f052 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:36.737224 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f4b5c-3dd7-4f34-91f4-e9f222b9f052 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:36.737224 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f4b5c-3dd7-4f34-91f4-e9f222b9f052 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:36.751249 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 152/301] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:36.754160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f4b5c-3dd7-4f34-91f4-e9f222b9f052 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:36.833176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d211ab-cea9-42c4-a95e-00c03c65677c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:36.860334 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f4b5c-3dd7-4f34-91f4-e9f222b9f052 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:36.869211 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2b106-91fa-426c-a4bf-6cfc3b430812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:36.870002 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2b106-91fa-426c-a4bf-6cfc3b430812 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:36.870287 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2b106-91fa-426c-a4bf-6cfc3b430812 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:36.870556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2b106-91fa-426c-a4bf-6cfc3b430812 None admin] PATH_INFO: `/v3/groups` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:36.871528 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2b106-91fa-426c-a4bf-6cfc3b430812 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:36.872830 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2b106-91fa-426c-a4bf-6cfc3b430812 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:36.898183 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ede827-88e0-4975-afe2-5e71e3d057ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:36.899536 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ede827-88e0-4975-afe2-5e71e3d057ae None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:36.900016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ede827-88e0-4975-afe2-5e71e3d057ae None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:36.900511 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ede827-88e0-4975-afe2-5e71e3d057ae None admin] PATH_INFO: `/v3/users/9a0a877a1295411d8b949948053a57f1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:36.913986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ede827-88e0-4975-afe2-5e71e3d057ae None admin] RBAC: Authorizing `identity:get_user(user_id=9a0a877a1295411d8b949948053a57f1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:36.915422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ede827-88e0-4975-afe2-5e71e3d057ae None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:36.917116 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 153/302] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:36 2022] GET /identity/v3/users/9a0a877a1295411d8b949948053a57f1 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:36.921383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c4cef-33ca-49e3-b34b-2d145578c33b None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:36.921660 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c4cef-33ca-49e3-b34b-2d145578c33b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:36.925092 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c4cef-33ca-49e3-b34b-2d145578c33b None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:36.925092 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 154/303] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:36.925092 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c4cef-33ca-49e3-b34b-2d145578c33b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:36.940648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:36.946013 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 150/304] 10.222.0.17 () {60 vars in 1134 bytes} [Tue Jun 7 08:19:36 2022] POST /identity/v3/groups => generated 229 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:19:36.947263 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385a974-3eec-4b6e-87ed-3e472a4df00e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:36.949615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385a974-3eec-4b6e-87ed-3e472a4df00e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:36.949615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385a974-3eec-4b6e-87ed-3e472a4df00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:36.976319 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-6385a974-3eec-4b6e-87ed-3e472a4df00e None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:36.977800 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6385a974-3eec-4b6e-87ed-3e472a4df00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:37.039957 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:37.055123 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c4cef-33ca-49e3-b34b-2d145578c33b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:37.070273 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 151/305] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:37.097334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385a974-3eec-4b6e-87ed-3e472a4df00e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:37.105224 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ef2d7-b144-429b-8f2f-1661525ccc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:37.105897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ef2d7-b144-429b-8f2f-1661525ccc18 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:37.106179 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ef2d7-b144-429b-8f2f-1661525ccc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:37.106468 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ef2d7-b144-429b-8f2f-1661525ccc18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:37.107299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ef2d7-b144-429b-8f2f-1661525ccc18 None admin] RBAC: Adding query filter params (role.id=5733a653e21248a7ab8ff7624b5f8ce9, user.id=9a0a877a1295411d8b949948053a57f1, scope.system=all) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:37.108759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ef2d7-b144-429b-8f2f-1661525ccc18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:37.110274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ef2d7-b144-429b-8f2f-1661525ccc18 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:37.117668 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 155/306] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Jun 7 08:19:36 2022] GET /identity/v3/role_assignments?user.id=9a0a877a1295411d8b949948053a57f1&scope.system=all&role.id=5733a653e21248a7ab8ff7624b5f8ce9 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:37.226624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385a974-3eec-4b6e-87ed-3e472a4df00e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:37.266116 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b549654-6c0e-468a-89a1-35539ca68239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:37.266947 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b549654-6c0e-468a-89a1-35539ca68239 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:37.267227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b549654-6c0e-468a-89a1-35539ca68239 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:37.267505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b549654-6c0e-468a-89a1-35539ca68239 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:37.280818 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b549654-6c0e-468a-89a1-35539ca68239 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:37.289568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-8b549654-6c0e-468a-89a1-35539ca68239 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:37.289568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:37.289568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:37.289568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:37.289568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:37.289568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:37.289568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:37.289568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:37.289568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:37.289568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:37.289568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:37.289568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:37.289568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:37.289568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:37.289568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:37.289568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:37.290583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:37.290583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:37.290583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:37.290583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:37.290583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:37.290583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:37.290583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:37.290583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:37.290583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:37.290583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:37.290583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:37.290583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:37.290583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:37.290583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:37.290583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:37.290583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:37.291450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:37.291450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:37.291450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:37.291450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:37.291450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:37.291450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:37.291450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:37.291450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:37.291450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:37.291450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:37.291450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:37.291450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:37.291450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 152/307] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:19:37.297761 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ef2d7-b144-429b-8f2f-1661525ccc18 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:37.380697 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:37.403706 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ef2d7-b144-429b-8f2f-1661525ccc18 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:37.443198 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9e06c-bacb-41e7-9c5c-eddc7515a0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:37.444696 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e06c-bacb-41e7-9c5c-eddc7515a0fb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:37.445182 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e06c-bacb-41e7-9c5c-eddc7515a0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:37.445631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e06c-bacb-41e7-9c5c-eddc7515a0fb None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:37.446589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9e06c-bacb-41e7-9c5c-eddc7515a0fb None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:37.447073 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9e06c-bacb-41e7-9c5c-eddc7515a0fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:37.448750 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9e06c-bacb-41e7-9c5c-eddc7515a0fb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:37.458512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 156/308] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:37.464726 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b549654-6c0e-468a-89a1-35539ca68239 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:37.502596 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:37.571848 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b549654-6c0e-468a-89a1-35539ca68239 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:37.606367 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec83fc0-87b2-4fdc-85a4-e8291fcc5f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:37.607558 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec83fc0-87b2-4fdc-85a4-e8291fcc5f78 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:37.608578 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec83fc0-87b2-4fdc-85a4-e8291fcc5f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:37.609106 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec83fc0-87b2-4fdc-85a4-e8291fcc5f78 None admin] PATH_INFO: `/v3/users/7b2d0d20f4f3479e8c7fbaee2f25560e` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:37.610491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec83fc0-87b2-4fdc-85a4-e8291fcc5f78 None admin] RBAC: Authorizing `identity:get_user(user_id=7b2d0d20f4f3479e8c7fbaee2f25560e)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:37.612130 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec83fc0-87b2-4fdc-85a4-e8291fcc5f78 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:37.614283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 153/309] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:37 2022] GET /identity/v3/users/7b2d0d20f4f3479e8c7fbaee2f25560e => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:37.620001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9e06c-bacb-41e7-9c5c-eddc7515a0fb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:37.724892 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9e06c-bacb-41e7-9c5c-eddc7515a0fb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:37.760519 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c248df2-1ee4-4946-845c-9292cfbaad84 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:37.760820 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c248df2-1ee4-4946-845c-9292cfbaad84 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:37.761122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c248df2-1ee4-4946-845c-9292cfbaad84 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:37.762606 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 154/310] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:37.763131 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15707be2-5a3e-40d2-b229-87d42700ebee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:37.764000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15707be2-5a3e-40d2-b229-87d42700ebee None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:37.764866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15707be2-5a3e-40d2-b229-87d42700ebee None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:37.765203 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15707be2-5a3e-40d2-b229-87d42700ebee None admin] PATH_INFO: `/v3/system/users/7b2d0d20f4f3479e8c7fbaee2f25560e/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:37.766775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15707be2-5a3e-40d2-b229-87d42700ebee None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7b2d0d20f4f3479e8c7fbaee2f25560e, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:37.769221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15707be2-5a3e-40d2-b229-87d42700ebee None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:37.769345 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c596f8-dc27-4325-aa30-026e3ee1d979 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:37.769526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c596f8-dc27-4325-aa30-026e3ee1d979 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:37.769803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c596f8-dc27-4325-aa30-026e3ee1d979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:37.787697 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d0c596f8-dc27-4325-aa30-026e3ee1d979 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:37.788878 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d0c596f8-dc27-4325-aa30-026e3ee1d979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:37.837833 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 157/311] 10.222.0.17 () {58 vars in 1334 bytes} [Tue Jun 7 08:19:37 2022] PUT /identity/v3/system/users/7b2d0d20f4f3479e8c7fbaee2f25560e/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:19:37.873209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 155/312] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:37.893118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15707be2-5a3e-40d2-b229-87d42700ebee None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:37.898918 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:38.016034 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15707be2-5a3e-40d2-b229-87d42700ebee None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:38.054636 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ac204-d81f-4adc-9577-c32c8ea27c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:38.055440 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ac204-d81f-4adc-9577-c32c8ea27c67 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:38.055720 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ac204-d81f-4adc-9577-c32c8ea27c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:38.055992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ac204-d81f-4adc-9577-c32c8ea27c67 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:38.066811 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ac204-d81f-4adc-9577-c32c8ea27c67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:38.074534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-609ac204-d81f-4adc-9577-c32c8ea27c67 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:19:38.074534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:38.074534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:38.074534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:38.074534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:38.074534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:38.074534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:38.074534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:38.074534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:38.074534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:38.074534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:38.074534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:38.074534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:38.074534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:38.074534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:38.074534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:38.075406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:38.075406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:38.075406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:38.075406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:38.075406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:38.075406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:38.075406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:38.075406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:38.075406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:38.075406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:38.075406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:38.075406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:38.075406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:38.075406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:38.075406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:38.075406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:38.076312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:38.076312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:38.076312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:38.076312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:38.076312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:38.076312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:38.076312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:38.076312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:38.076312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:38.076312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:38.076312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:19:38.076312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:38.076312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 158/313] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:38.085290 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c596f8-dc27-4325-aa30-026e3ee1d979 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:38.120930 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:38.190930 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c596f8-dc27-4325-aa30-026e3ee1d979 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:38.231484 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd8a0e-c4f1-4162-8d1b-541457fc51c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:38.232963 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd8a0e-c4f1-4162-8d1b-541457fc51c5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:38.233471 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd8a0e-c4f1-4162-8d1b-541457fc51c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:38.233942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd8a0e-c4f1-4162-8d1b-541457fc51c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:38.234895 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd8a0e-c4f1-4162-8d1b-541457fc51c5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:38.235397 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd8a0e-c4f1-4162-8d1b-541457fc51c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:38.236975 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd8a0e-c4f1-4162-8d1b-541457fc51c5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:38.245635 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 156/314] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:38.253121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ac204-d81f-4adc-9577-c32c8ea27c67 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:38.324297 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:38.366475 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ac204-d81f-4adc-9577-c32c8ea27c67 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:38.405101 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d1931-8689-442b-bbd5-e6a093b3b7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:38.405928 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d1931-8689-442b-bbd5-e6a093b3b7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:38.406208 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d1931-8689-442b-bbd5-e6a093b3b7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:38.406481 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d1931-8689-442b-bbd5-e6a093b3b7a9 None admin] PATH_INFO: `/v3/users/3ba4873935a64249b2ae9c8348646ce6` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:38.407670 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d1931-8689-442b-bbd5-e6a093b3b7a9 None admin] RBAC: Authorizing `identity:get_user(user_id=3ba4873935a64249b2ae9c8348646ce6)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:38.409137 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d1931-8689-442b-bbd5-e6a093b3b7a9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:38.411341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 159/315] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:38 2022] GET /identity/v3/users/3ba4873935a64249b2ae9c8348646ce6 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:38.416759 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd8a0e-c4f1-4162-8d1b-541457fc51c5 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:38.519010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:38.524584 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd8a0e-c4f1-4162-8d1b-541457fc51c5 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:38.558973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab98f3-6ed7-4613-acab-e17381ff1114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:38.559998 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab98f3-6ed7-4613-acab-e17381ff1114 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:38.560774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab98f3-6ed7-4613-acab-e17381ff1114 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:38.561638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab98f3-6ed7-4613-acab-e17381ff1114 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:38.630229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab98f3-6ed7-4613-acab-e17381ff1114 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:38.631760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab98f3-6ed7-4613-acab-e17381ff1114 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:38.729381 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:38.732807 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 157/316] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:38.739758 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d1931-8689-442b-bbd5-e6a093b3b7a9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:38.843844 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d1931-8689-442b-bbd5-e6a093b3b7a9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:38.873848 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:38.881140 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3da59-f203-42ed-add5-445b5bbd8416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:38.882103 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3da59-f203-42ed-add5-445b5bbd8416 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:38.882398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3da59-f203-42ed-add5-445b5bbd8416 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:38.882701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3da59-f203-42ed-add5-445b5bbd8416 None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:38.883873 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3da59-f203-42ed-add5-445b5bbd8416 None admin] RBAC: Authorizing `identity:get_project(project_id=914fc5f0eaf745279782c417df7acf69)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:38.885336 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3da59-f203-42ed-add5-445b5bbd8416 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:38.887381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 160/317] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:38 2022] GET /identity/v3/projects/914fc5f0eaf745279782c417df7acf69 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:38.893180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab98f3-6ed7-4613-acab-e17381ff1114 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:38.995723 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab98f3-6ed7-4613-acab-e17381ff1114 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:39.032812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53638b53-03f2-4296-b77f-5df412b83ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:39.033518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53638b53-03f2-4296-b77f-5df412b83ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:39.033799 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53638b53-03f2-4296-b77f-5df412b83ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:39.034073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53638b53-03f2-4296-b77f-5df412b83ef4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:39.034924 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53638b53-03f2-4296-b77f-5df412b83ef4 None admin] RBAC: Adding query filter params (role.id=5733a653e21248a7ab8ff7624b5f8ce9, scope.project.id=914fc5f0eaf745279782c417df7acf69, user.id=3ba4873935a64249b2ae9c8348646ce6) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:39.035243 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53638b53-03f2-4296-b77f-5df412b83ef4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:39.036865 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53638b53-03f2-4296-b77f-5df412b83ef4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:39.043937 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 158/318] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:19:38 2022] GET /identity/v3/role_assignments?user.id=3ba4873935a64249b2ae9c8348646ce6&scope.project.id=914fc5f0eaf745279782c417df7acf69&role.id=5733a653e21248a7ab8ff7624b5f8ce9 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:39.388755 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b958f5-febd-4b76-84e2-ed118c1c5bdb None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:39.389637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b958f5-febd-4b76-84e2-ed118c1c5bdb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:39.389824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b958f5-febd-4b76-84e2-ed118c1c5bdb None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:39.391344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 161/319] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:39.396335 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf9b7f-9001-4a72-a7ff-ca939c86d0a7 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:39.396757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf9b7f-9001-4a72-a7ff-ca939c86d0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:39.396908 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf9b7f-9001-4a72-a7ff-ca939c86d0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:39.414120 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-0bcf9b7f-9001-4a72-a7ff-ca939c86d0a7 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:39.415305 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0bcf9b7f-9001-4a72-a7ff-ca939c86d0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:39.472822 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:39.476693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317eb2f2-6748-4fff-a38b-6ccfcbc57113 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:39.476985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317eb2f2-6748-4fff-a38b-6ccfcbc57113 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:39.477286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317eb2f2-6748-4fff-a38b-6ccfcbc57113 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:39.478644 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 162/320] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:19:39.483434 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd051b-d35a-4734-91a3-b2dfb4e9ee68 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:39.483714 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd051b-d35a-4734-91a3-b2dfb4e9ee68 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:39.484021 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd051b-d35a-4734-91a3-b2dfb4e9ee68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:39.484493 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 159/321] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:39.500741 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-22fd051b-d35a-4734-91a3-b2dfb4e9ee68 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:39.501933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-22fd051b-d35a-4734-91a3-b2dfb4e9ee68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:39.509576 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf9b7f-9001-4a72-a7ff-ca939c86d0a7 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:39.575202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 163/322] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:39.597616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd051b-d35a-4734-91a3-b2dfb4e9ee68 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:39.615065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf9b7f-9001-4a72-a7ff-ca939c86d0a7 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:39.650458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e8bbd-d038-46bf-893e-fb9b561a79d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:39.651165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e8bbd-d038-46bf-893e-fb9b561a79d5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:39.651444 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e8bbd-d038-46bf-893e-fb9b561a79d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:39.651714 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e8bbd-d038-46bf-893e-fb9b561a79d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:39.652861 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e8bbd-d038-46bf-893e-fb9b561a79d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:39.654329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e8bbd-d038-46bf-893e-fb9b561a79d5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:39.660389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 160/323] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:19:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:39.661360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e8bbd-d038-46bf-893e-fb9b561a79d5 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:39.703913 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd051b-d35a-4734-91a3-b2dfb4e9ee68 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:39.725265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:39.734200 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:39.739894 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce981f8-c202-48be-be0d-27ecd20a9003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:39.740686 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce981f8-c202-48be-be0d-27ecd20a9003 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:39.740966 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce981f8-c202-48be-be0d-27ecd20a9003 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:39.741265 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce981f8-c202-48be-be0d-27ecd20a9003 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:39.752955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce981f8-c202-48be-be0d-27ecd20a9003 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:39.760637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5ce981f8-c202-48be-be0d-27ecd20a9003 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:19:39.760637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:39.760637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:39.760637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:39.760637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:39.760637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:39.760637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:39.760637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:39.760637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:39.760637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:39.760637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:39.760637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:39.760637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:39.760637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:39.760637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:39.760637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:39.761717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:39.761717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:39.761717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:39.761717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:39.761717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:39.761717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:39.761717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:39.761717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:39.761717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:39.761717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:39.761717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:39.761717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:39.761717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:39.761717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:39.761717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:39.761717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:39.762493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:39.762493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:39.762493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:39.762493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:39.762493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:39.762493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:39.762493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:39.762493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:39.762493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:39.762493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:39.762493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:19:39.762493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:39.762493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 164/324] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:19:39.768004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce981f8-c202-48be-be0d-27ecd20a9003 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:39.774899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e8bbd-d038-46bf-893e-fb9b561a79d5 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:39.812570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf67f0-3ae3-4ee6-8025-9345884a49fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:39.813095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf67f0-3ae3-4ee6-8025-9345884a49fe None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:39.813373 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf67f0-3ae3-4ee6-8025-9345884a49fe None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:39.813932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf67f0-3ae3-4ee6-8025-9345884a49fe None admin] PATH_INFO: `/v3/groups` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:39.814435 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf67f0-3ae3-4ee6-8025-9345884a49fe None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:39.815866 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf67f0-3ae3-4ee6-8025-9345884a49fe None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:39.871297 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce981f8-c202-48be-be0d-27ecd20a9003 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:39.899827 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 161/325] 10.222.0.17 () {60 vars in 1134 bytes} [Tue Jun 7 08:19:39 2022] POST /identity/v3/groups => generated 226 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:19:39.906206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c8374-19b8-4de5-9d41-d725de0b5b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:39.906878 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c8374-19b8-4de5-9d41-d725de0b5b6e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:39.907161 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c8374-19b8-4de5-9d41-d725de0b5b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:39.907434 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c8374-19b8-4de5-9d41-d725de0b5b6e None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:39.908247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c8374-19b8-4de5-9d41-d725de0b5b6e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:39.908644 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c8374-19b8-4de5-9d41-d725de0b5b6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:39.910198 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c8374-19b8-4de5-9d41-d725de0b5b6e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:39.925908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 165/326] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:39.927254 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf67f0-3ae3-4ee6-8025-9345884a49fe None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:40.040401 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf67f0-3ae3-4ee6-8025-9345884a49fe None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:40.058946 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:40.077705 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63997b37-1803-4306-a758-5581beac1386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:40.078657 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63997b37-1803-4306-a758-5581beac1386 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:40.078983 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63997b37-1803-4306-a758-5581beac1386 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:40.079259 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63997b37-1803-4306-a758-5581beac1386 None admin] PATH_INFO: `/v3/users/9a0a877a1295411d8b949948053a57f1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:40.080513 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63997b37-1803-4306-a758-5581beac1386 None admin] RBAC: Authorizing `identity:get_user(user_id=9a0a877a1295411d8b949948053a57f1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:40.082311 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63997b37-1803-4306-a758-5581beac1386 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:40.088696 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 162/327] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:39 2022] GET /identity/v3/users/9a0a877a1295411d8b949948053a57f1 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:40.090019 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c8374-19b8-4de5-9d41-d725de0b5b6e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:40.149887 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:40.205833 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c8374-19b8-4de5-9d41-d725de0b5b6e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:40.241989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be501f-933e-4c60-8a4f-25862b536362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:40.242855 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be501f-933e-4c60-8a4f-25862b536362 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:40.243133 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be501f-933e-4c60-8a4f-25862b536362 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:40.243405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be501f-933e-4c60-8a4f-25862b536362 None admin] PATH_INFO: `/v3/system/users/9a0a877a1295411d8b949948053a57f1/roles/5733a653e21248a7ab8ff7624b5f8ce9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:40.244885 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be501f-933e-4c60-8a4f-25862b536362 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9a0a877a1295411d8b949948053a57f1, role_id=5733a653e21248a7ab8ff7624b5f8ce9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:40.246290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be501f-933e-4c60-8a4f-25862b536362 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:40.259118 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76cb9a-acd0-4e9c-94c3-3f05e605f6e8 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:40.259407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76cb9a-acd0-4e9c-94c3-3f05e605f6e8 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:40.259684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76cb9a-acd0-4e9c-94c3-3f05e605f6e8 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:40.261083 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 163/328] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:19:40.267104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9bc5c-03c3-4145-a0ab-32d44f20a880 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:40.267385 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9bc5c-03c3-4145-a0ab-32d44f20a880 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:40.267655 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9bc5c-03c3-4145-a0ab-32d44f20a880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:40.285694 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c9f9bc5c-03c3-4145-a0ab-32d44f20a880 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:40.286967 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c9f9bc5c-03c3-4145-a0ab-32d44f20a880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:40.312906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 166/329] 10.222.0.17 () {58 vars in 1334 bytes} [Tue Jun 7 08:19:40 2022] PUT /identity/v3/system/users/9a0a877a1295411d8b949948053a57f1/roles/5733a653e21248a7ab8ff7624b5f8ce9 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:19:40.366795 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 164/330] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:40.391487 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be501f-933e-4c60-8a4f-25862b536362 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:40.515121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be501f-933e-4c60-8a4f-25862b536362 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:40.533263 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:40.554973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d96be-c58c-4625-ad1e-4f0f553ebc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:40.555819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d96be-c58c-4625-ad1e-4f0f553ebc25 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:40.556097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d96be-c58c-4625-ad1e-4f0f553ebc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:40.556368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d96be-c58c-4625-ad1e-4f0f553ebc25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:40.568061 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d96be-c58c-4625-ad1e-4f0f553ebc25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:40.577867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e53d96be-c58c-4625-ad1e-4f0f553ebc25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:40.577867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:40.577867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:40.577867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:40.577867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:40.577867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:40.577867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:40.577867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:40.577867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:40.577867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:40.577867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:40.577867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:40.577867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:40.577867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:40.577867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:40.577867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:40.578774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:40.578774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:40.578774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:40.578774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:40.578774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:40.578774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:40.578774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:40.578774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:40.578774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:40.578774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:40.578774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:40.578774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:40.578774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:40.578774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:40.578774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:40.578774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:40.579548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:40.579548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:40.579548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:40.579548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:40.579548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:40.579548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:40.579548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:40.579548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:40.579548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:40.579548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:40.579548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:40.579548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:40.579548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 167/331] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:40.585177 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9bc5c-03c3-4145-a0ab-32d44f20a880 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:40.691010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9bc5c-03c3-4145-a0ab-32d44f20a880 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:40.708576 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:40.727905 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e3b00-4b00-4054-b88a-566542a6bf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:40.728572 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e3b00-4b00-4054-b88a-566542a6bf9a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:40.728894 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e3b00-4b00-4054-b88a-566542a6bf9a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:40.729203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e3b00-4b00-4054-b88a-566542a6bf9a None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:40.729948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e3b00-4b00-4054-b88a-566542a6bf9a None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:40.730249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e3b00-4b00-4054-b88a-566542a6bf9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:40.731670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e3b00-4b00-4054-b88a-566542a6bf9a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:40.741090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 165/332] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:40.746592 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d96be-c58c-4625-ad1e-4f0f553ebc25 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:40.862895 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d96be-c58c-4625-ad1e-4f0f553ebc25 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:40.900743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774c93b-452d-4685-85d3-3d21fcc91f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:40.901668 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774c93b-452d-4685-85d3-3d21fcc91f76 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:40.901941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774c93b-452d-4685-85d3-3d21fcc91f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:40.902208 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774c93b-452d-4685-85d3-3d21fcc91f76 None admin] PATH_INFO: `/v3/users/7b2d0d20f4f3479e8c7fbaee2f25560e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:40.903391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3774c93b-452d-4685-85d3-3d21fcc91f76 None admin] RBAC: Authorizing `identity:get_user(user_id=7b2d0d20f4f3479e8c7fbaee2f25560e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:40.905407 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3774c93b-452d-4685-85d3-3d21fcc91f76 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:40.907019 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 168/333] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:40 2022] GET /identity/v3/users/7b2d0d20f4f3479e8c7fbaee2f25560e => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:40.912954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e3b00-4b00-4054-b88a-566542a6bf9a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:41.017568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e3b00-4b00-4054-b88a-566542a6bf9a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:41.055516 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174df9e-9351-4fd4-8677-e34c4b999459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:41.056280 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174df9e-9351-4fd4-8677-e34c4b999459 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:41.056564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174df9e-9351-4fd4-8677-e34c4b999459 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:41.056884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174df9e-9351-4fd4-8677-e34c4b999459 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:41.057746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174df9e-9351-4fd4-8677-e34c4b999459 None admin] RBAC: Adding query filter params (role.id=67111011e94c4e08928dd5e6b7a46af1, user.id=7b2d0d20f4f3479e8c7fbaee2f25560e, scope.system=all) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:41.058055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174df9e-9351-4fd4-8677-e34c4b999459 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:41.059475 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174df9e-9351-4fd4-8677-e34c4b999459 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:41.072854 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 166/334] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Jun 7 08:19:40 2022] GET /identity/v3/role_assignments?user.id=7b2d0d20f4f3479e8c7fbaee2f25560e&scope.system=all&role.id=67111011e94c4e08928dd5e6b7a46af1 => generated 510 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:42.510776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5f9b7-01a1-4360-8bfc-f54b68854453 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:42.511278 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5f9b7-01a1-4360-8bfc-f54b68854453 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:42.511554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5f9b7-01a1-4360-8bfc-f54b68854453 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:42.513219 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 169/335] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:42.526651 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e70e7-d514-4e78-8171-7c23390c9f91 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:42.527148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e70e7-d514-4e78-8171-7c23390c9f91 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:42.527221 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e70e7-d514-4e78-8171-7c23390c9f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:42.550240 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-f33e70e7-d514-4e78-8171-7c23390c9f91 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:42.557374 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f33e70e7-d514-4e78-8171-7c23390c9f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:42.591684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:42.636223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 167/336] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:42.683614 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5f9b7-01a1-4360-8bfc-f54b68854453 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:42.733900 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:42.799764 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5f9b7-01a1-4360-8bfc-f54b68854453 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:42.845322 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2c44a-47e7-4051-8a82-1fff4ddd89b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:42.846120 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2c44a-47e7-4051-8a82-1fff4ddd89b3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:42.846402 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2c44a-47e7-4051-8a82-1fff4ddd89b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:42.846673 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2c44a-47e7-4051-8a82-1fff4ddd89b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:42.858765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c2c44a-47e7-4051-8a82-1fff4ddd89b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:42.866565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-13c2c44a-47e7-4051-8a82-1fff4ddd89b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:42.866565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:42.866565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:42.866565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:42.866565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:42.866565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:42.866565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:42.866565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:42.866565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:42.866565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:42.866565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:42.866565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:42.866565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:42.866565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:42.866565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:42.866565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:42.867463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:42.867463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:42.867463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:42.867463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:42.867463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:42.867463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:42.867463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:42.867463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:42.867463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:42.867463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:42.867463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:42.867463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:42.867463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:42.867463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:42.867463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:42.867463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:42.868222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:42.868222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:42.868222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:42.868222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:42.868222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:42.868222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:42.868222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:42.868222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:42.868222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:42.868222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:42.868222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:42.868222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:42.868222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 170/337] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:42.875903 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e70e7-d514-4e78-8171-7c23390c9f91 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:42.982643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e70e7-d514-4e78-8171-7c23390c9f91 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:43.016307 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc372329-91e7-4281-8668-a78a141526f8 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:43.016617 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc372329-91e7-4281-8668-a78a141526f8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:43.016858 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc372329-91e7-4281-8668-a78a141526f8 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:43.018231 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 171/338] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:43.026357 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04ac51-c725-46fb-8efe-461552083216 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:43.026631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04ac51-c725-46fb-8efe-461552083216 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:43.026903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04ac51-c725-46fb-8efe-461552083216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:43.059261 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6c04ac51-c725-46fb-8efe-461552083216 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:43.060367 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6c04ac51-c725-46fb-8efe-461552083216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:43.064039 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d6eb1-4041-4269-a2c4-6eef6d2e09ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:43.064707 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d6eb1-4041-4269-a2c4-6eef6d2e09ac None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:43.064983 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d6eb1-4041-4269-a2c4-6eef6d2e09ac None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:43.065274 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d6eb1-4041-4269-a2c4-6eef6d2e09ac None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:43.066019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d6eb1-4041-4269-a2c4-6eef6d2e09ac None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:43.066316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d6eb1-4041-4269-a2c4-6eef6d2e09ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:43.067678 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d6eb1-4041-4269-a2c4-6eef6d2e09ac None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:43.083635 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 168/339] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:43.090008 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d6eb1-4041-4269-a2c4-6eef6d2e09ac None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:43.129371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:43.195013 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 172/340] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:43.196357 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd87be5-99ac-4392-9c21-9b50b1bc710b None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:43.196878 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd87be5-99ac-4392-9c21-9b50b1bc710b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:43.197388 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd87be5-99ac-4392-9c21-9b50b1bc710b None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:43.198860 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 173/341] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:43.210478 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb589de-31f1-46f8-ac27-0f51201dd035 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:43.211088 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb589de-31f1-46f8-ac27-0f51201dd035 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:43.211562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb589de-31f1-46f8-ac27-0f51201dd035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:43.240009 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ceb589de-31f1-46f8-ac27-0f51201dd035 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:43.241585 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ceb589de-31f1-46f8-ac27-0f51201dd035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:43.268435 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d6eb1-4041-4269-a2c4-6eef6d2e09ac None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:43.274385 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:43.371813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 174/342] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:43.371813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb589de-31f1-46f8-ac27-0f51201dd035 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:43.372151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55af1c6-b810-4295-91cf-1710d72c2d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:43.372151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55af1c6-b810-4295-91cf-1710d72c2d51 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:43.372151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55af1c6-b810-4295-91cf-1710d72c2d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:43.372151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55af1c6-b810-4295-91cf-1710d72c2d51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:43.463232 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb589de-31f1-46f8-ac27-0f51201dd035 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:43.476823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55af1c6-b810-4295-91cf-1710d72c2d51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:43.478598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55af1c6-b810-4295-91cf-1710d72c2d51 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:43.514763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:43.537731 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091f5bb-56df-4d63-8ce4-2094a781f309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:43.538817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091f5bb-56df-4d63-8ce4-2094a781f309 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:43.539168 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091f5bb-56df-4d63-8ce4-2094a781f309 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:43.539509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091f5bb-56df-4d63-8ce4-2094a781f309 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:43.562043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1091f5bb-56df-4d63-8ce4-2094a781f309 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:43.572792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1091f5bb-56df-4d63-8ce4-2094a781f309 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:19:43.572792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:43.572792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:43.572792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:43.572792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:43.572792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:43.572792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:43.572792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:43.572792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:43.572792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:43.572792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:43.572792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:43.572792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:43.572792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:43.572792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:43.572792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:43.573756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:43.573756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:43.573756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:43.573756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:43.573756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:43.573756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:43.573756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:43.573756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:43.573756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:43.573756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:43.573756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:43.573756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:43.573756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:43.573756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:43.573756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:43.573756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:43.574589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:43.574589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:43.574589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:43.574589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:43.574589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:43.574589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:43.574589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:43.574589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:43.574589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:43.574589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:43.574589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 08:19:43.574589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:43.574589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 175/343] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:43.575255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091f5bb-56df-4d63-8ce4-2094a781f309 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:43.604958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:43.633197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 169/344] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:43.634095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55af1c6-b810-4295-91cf-1710d72c2d51 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:43.740277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091f5bb-56df-4d63-8ce4-2094a781f309 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:43.753563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55af1c6-b810-4295-91cf-1710d72c2d51 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:43.788347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:43.797906 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8439043-4567-4b2a-a52f-33e19c822866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:43.799485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8439043-4567-4b2a-a52f-33e19c822866 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:43.799973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8439043-4567-4b2a-a52f-33e19c822866 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:43.800409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8439043-4567-4b2a-a52f-33e19c822866 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:43.801343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8439043-4567-4b2a-a52f-33e19c822866 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:43.807197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8439043-4567-4b2a-a52f-33e19c822866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:43.807197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8439043-4567-4b2a-a52f-33e19c822866 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:43.818361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485d0f3-a52a-4bc3-b4a2-8a6db3f30f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:43.819180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485d0f3-a52a-4bc3-b4a2-8a6db3f30f19 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:43.819504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485d0f3-a52a-4bc3-b4a2-8a6db3f30f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:43.819779 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485d0f3-a52a-4bc3-b4a2-8a6db3f30f19 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:43.820081 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 170/345] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:43.821134 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8439043-4567-4b2a-a52f-33e19c822866 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:43.832955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485d0f3-a52a-4bc3-b4a2-8a6db3f30f19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:43.843793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-3485d0f3-a52a-4bc3-b4a2-8a6db3f30f19 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:43.843793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:43.843793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:43.843793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:43.843793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:43.843793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:43.843793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:43.843793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:43.843793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:43.843793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:43.843793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:43.843793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:43.843793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:43.843793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:43.843793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:43.843793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:43.844707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:43.844707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:43.844707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:43.844707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:43.844707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:43.844707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:43.844707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:43.844707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:43.844707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:43.844707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:43.844707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:43.844707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:43.844707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:43.844707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:43.844707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:43.844707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:43.845543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:43.845543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:43.845543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:43.845543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:43.845543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:43.845543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:43.845543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:43.845543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:43.845543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:43.845543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:43.845543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:43.845543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:43.845543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 176/346] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:19:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:43.846218 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485d0f3-a52a-4bc3-b4a2-8a6db3f30f19 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:43.942502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:43.960580 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485d0f3-a52a-4bc3-b4a2-8a6db3f30f19 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:43.997484 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62356adf-998b-4155-bc9e-5c17446f88ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:43.999154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62356adf-998b-4155-bc9e-5c17446f88ae None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:43.999589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62356adf-998b-4155-bc9e-5c17446f88ae None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:44.000024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62356adf-998b-4155-bc9e-5c17446f88ae None admin] PATH_INFO: `/v3/users/9a0a877a1295411d8b949948053a57f1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:44.001408 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62356adf-998b-4155-bc9e-5c17446f88ae None admin] RBAC: Authorizing `identity:get_user(user_id=9a0a877a1295411d8b949948053a57f1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:44.003025 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62356adf-998b-4155-bc9e-5c17446f88ae None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:44.004799 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 177/347] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:19:43 2022] GET /identity/v3/users/9a0a877a1295411d8b949948053a57f1 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:44.005721 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62356adf-998b-4155-bc9e-5c17446f88ae None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:44.037839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8439043-4567-4b2a-a52f-33e19c822866 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:44.101722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef462532-817d-43ae-9488-e4d1c6338c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:44.102612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef462532-817d-43ae-9488-e4d1c6338c00 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:44.102905 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef462532-817d-43ae-9488-e4d1c6338c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:44.103176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef462532-817d-43ae-9488-e4d1c6338c00 None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:44.104425 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef462532-817d-43ae-9488-e4d1c6338c00 None admin] RBAC: Authorizing `identity:get_project(project_id=fe9516226a724cf688d3d23e1392161d)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:44.106094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef462532-817d-43ae-9488-e4d1c6338c00 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:44.107973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 171/348] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:43 2022] GET /identity/v3/projects/fe9516226a724cf688d3d23e1392161d => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:44.112378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef462532-817d-43ae-9488-e4d1c6338c00 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:44.115303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62356adf-998b-4155-bc9e-5c17446f88ae None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:44.151603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bb984-97b5-4324-9680-dd85b5f02421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:44.152512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bb984-97b5-4324-9680-dd85b5f02421 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:44.153015 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bb984-97b5-4324-9680-dd85b5f02421 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:44.153454 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bb984-97b5-4324-9680-dd85b5f02421 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:44.154359 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bb984-97b5-4324-9680-dd85b5f02421 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:44.154842 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bb984-97b5-4324-9680-dd85b5f02421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:44.156366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bb984-97b5-4324-9680-dd85b5f02421 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:44.165151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 178/349] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:19:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:44.166157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bb984-97b5-4324-9680-dd85b5f02421 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:44.192153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:44.213842 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:44.249758 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef462532-817d-43ae-9488-e4d1c6338c00 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:44.283820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bb984-97b5-4324-9680-dd85b5f02421 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:44.286051 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e18090-fda4-4768-8773-36eb6408e3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:44.286845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e18090-fda4-4768-8773-36eb6408e3bc None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:44.287003 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e18090-fda4-4768-8773-36eb6408e3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:44.287285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e18090-fda4-4768-8773-36eb6408e3bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:44.288324 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e18090-fda4-4768-8773-36eb6408e3bc None admin] RBAC: Adding query filter params (role.id=5733a653e21248a7ab8ff7624b5f8ce9, user.id=9a0a877a1295411d8b949948053a57f1, scope.system=all) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:44.288450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e18090-fda4-4768-8773-36eb6408e3bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:44.289918 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e18090-fda4-4768-8773-36eb6408e3bc None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:44.304893 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 172/350] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Jun 7 08:19:44 2022] GET /identity/v3/role_assignments?user.id=9a0a877a1295411d8b949948053a57f1&scope.system=all&role.id=5733a653e21248a7ab8ff7624b5f8ce9 => generated 510 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:44.310055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e18090-fda4-4768-8773-36eb6408e3bc None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:44.324551 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730cf4e2-aee6-436f-9e04-fe5b7a5897a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:44.325698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cf4e2-aee6-436f-9e04-fe5b7a5897a1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:44.326195 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cf4e2-aee6-436f-9e04-fe5b7a5897a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:44.327227 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cf4e2-aee6-436f-9e04-fe5b7a5897a1 None admin] PATH_INFO: `/v3/groups/730ea508c3fd4575af2609bb7f189405` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:44.334542 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730cf4e2-aee6-436f-9e04-fe5b7a5897a1 None admin] RBAC: Authorizing `identity:get_group(group_id=730ea508c3fd4575af2609bb7f189405)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:44.336335 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730cf4e2-aee6-436f-9e04-fe5b7a5897a1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:44.338275 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 179/351] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jun 7 08:19:44 2022] GET /identity/v3/groups/730ea508c3fd4575af2609bb7f189405 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:44.344228 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730cf4e2-aee6-436f-9e04-fe5b7a5897a1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:44.432294 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:44.456213 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730cf4e2-aee6-436f-9e04-fe5b7a5897a1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:44.458737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e18090-fda4-4768-8773-36eb6408e3bc None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:44.480499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:44.503098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6440f28-9b06-499e-a9f3-9820d4571ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:44.503760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6440f28-9b06-499e-a9f3-9820d4571ddb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:44.504046 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6440f28-9b06-499e-a9f3-9820d4571ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:44.504324 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6440f28-9b06-499e-a9f3-9820d4571ddb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:44.505206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6440f28-9b06-499e-a9f3-9820d4571ddb None admin] RBAC: Adding query filter params (group.id=730ea508c3fd4575af2609bb7f189405, role.id=67111011e94c4e08928dd5e6b7a46af1, scope.project.id=fe9516226a724cf688d3d23e1392161d) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:44.505523 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6440f28-9b06-499e-a9f3-9820d4571ddb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:44.506986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6440f28-9b06-499e-a9f3-9820d4571ddb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:44.509605 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65b630-0b2d-4db0-a41f-936b713702b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:44.510213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65b630-0b2d-4db0-a41f-936b713702b0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:44.510497 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65b630-0b2d-4db0-a41f-936b713702b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:44.510772 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65b630-0b2d-4db0-a41f-936b713702b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:44.526427 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 180/352] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jun 7 08:19:44 2022] GET /identity/v3/role_assignments?group.id=730ea508c3fd4575af2609bb7f189405&scope.project.id=fe9516226a724cf688d3d23e1392161d&role.id=67111011e94c4e08928dd5e6b7a46af1 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:44.590747 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed50d40-a566-4950-8360-88e95ca99041 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:44.591383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed50d40-a566-4950-8360-88e95ca99041 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:44.591383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed50d40-a566-4950-8360-88e95ca99041 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:44.593462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 181/353] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:19:44.599043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94269a5e-ec04-4e48-8abf-6e482c1368a2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:44.599345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94269a5e-ec04-4e48-8abf-6e482c1368a2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:44.599600 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94269a5e-ec04-4e48-8abf-6e482c1368a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:44.599849 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65b630-0b2d-4db0-a41f-936b713702b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:44.601501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65b630-0b2d-4db0-a41f-936b713702b0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:44.649134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-94269a5e-ec04-4e48-8abf-6e482c1368a2 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:44.650353 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-94269a5e-ec04-4e48-8abf-6e482c1368a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:44.720410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 173/354] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:44.726131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65b630-0b2d-4db0-a41f-936b713702b0 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:44.752300 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:44.838931 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 182/355] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:44.872480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65b630-0b2d-4db0-a41f-936b713702b0 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:44.880402 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94269a5e-ec04-4e48-8abf-6e482c1368a2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:44.917631 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d4135-c8c7-4971-b7c1-f2a9b83da1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:44.918616 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d4135-c8c7-4971-b7c1-f2a9b83da1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:44.918891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d4135-c8c7-4971-b7c1-f2a9b83da1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:44.919162 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d4135-c8c7-4971-b7c1-f2a9b83da1c9 None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:44.920718 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d4135-c8c7-4971-b7c1-f2a9b83da1c9 None admin] RBAC: Authorizing `identity:get_project(project_id=fe9516226a724cf688d3d23e1392161d)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:44.923024 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d4135-c8c7-4971-b7c1-f2a9b83da1c9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:44.925780 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 174/356] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:44 2022] GET /identity/v3/projects/fe9516226a724cf688d3d23e1392161d => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:44.931109 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d4135-c8c7-4971-b7c1-f2a9b83da1c9 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:44.976222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:45.009392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94269a5e-ec04-4e48-8abf-6e482c1368a2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:45.050561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37597145-a8d7-4194-a6e1-d416df1d9a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:45.051361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37597145-a8d7-4194-a6e1-d416df1d9a77 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:45.051639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37597145-a8d7-4194-a6e1-d416df1d9a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:45.051910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37597145-a8d7-4194-a6e1-d416df1d9a77 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:45.063310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37597145-a8d7-4194-a6e1-d416df1d9a77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:45.066765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:45.071144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-37597145-a8d7-4194-a6e1-d416df1d9a77 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:45.071144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:45.071144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:45.071144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:45.071144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:45.071144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:45.071144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:45.071144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:45.071144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:45.071144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:45.071144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:45.071144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:45.071144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:45.071144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:45.071144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:45.071144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:45.073268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:45.073268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:45.073268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:45.073268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:45.073268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:45.073268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:45.073268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:45.073268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:45.073268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:45.073268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:45.073268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:45.073268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:45.073268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:45.073268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:45.073268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:45.073268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:45.080757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:45.080757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:45.080757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:45.080757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:45.080757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:45.080757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:45.080757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:45.080757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:45.080757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:45.080757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:45.080757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:45.080757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:45.080757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 183/357] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:19:45.080757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37597145-a8d7-4194-a6e1-d416df1d9a77 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:45.106347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d4135-c8c7-4971-b7c1-f2a9b83da1c9 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:45.149605 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c059470-3e8f-4eef-abae-e121a93b153f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:45.150974 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c059470-3e8f-4eef-abae-e121a93b153f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:45.151622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c059470-3e8f-4eef-abae-e121a93b153f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:45.152098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c059470-3e8f-4eef-abae-e121a93b153f None admin] PATH_INFO: `/v3/groups/730ea508c3fd4575af2609bb7f189405` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:45.154531 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c059470-3e8f-4eef-abae-e121a93b153f None admin] RBAC: Authorizing `identity:get_group(group_id=730ea508c3fd4575af2609bb7f189405)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:45.156242 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c059470-3e8f-4eef-abae-e121a93b153f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:45.158579 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 175/358] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jun 7 08:19:44 2022] GET /identity/v3/groups/730ea508c3fd4575af2609bb7f189405 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:45.160577 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc1ae1-3ca9-492c-aabc-ab110498fbb9 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:45.161102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc1ae1-3ca9-492c-aabc-ab110498fbb9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:45.161957 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc1ae1-3ca9-492c-aabc-ab110498fbb9 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:45.164275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 176/359] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:45.165219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc1ae1-3ca9-492c-aabc-ab110498fbb9 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:45.203224 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37597145-a8d7-4194-a6e1-d416df1d9a77 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:45.246453 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:45.254082 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f011f30-adea-474f-a67e-9b516d14e773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:45.255615 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f011f30-adea-474f-a67e-9b516d14e773 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:45.256062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f011f30-adea-474f-a67e-9b516d14e773 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:45.256639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f011f30-adea-474f-a67e-9b516d14e773 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:45.257662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f011f30-adea-474f-a67e-9b516d14e773 None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:45.258124 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f011f30-adea-474f-a67e-9b516d14e773 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:45.271133 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f011f30-adea-474f-a67e-9b516d14e773 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:45.274402 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 184/360] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:45.276257 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bbd20-5495-44d3-a5bb-c913302d755b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:45.276965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bbd20-5495-44d3-a5bb-c913302d755b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:45.277407 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bbd20-5495-44d3-a5bb-c913302d755b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:45.300798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2b9bbd20-5495-44d3-a5bb-c913302d755b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:45.302815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2b9bbd20-5495-44d3-a5bb-c913302d755b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:45.317353 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc1ae1-3ca9-492c-aabc-ab110498fbb9 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:45.360762 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c784bb6-2d77-4d1b-8fd8-d2b1981d8fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:45.361949 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c784bb6-2d77-4d1b-8fd8-d2b1981d8fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:45.362451 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c784bb6-2d77-4d1b-8fd8-d2b1981d8fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:45.362915 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c784bb6-2d77-4d1b-8fd8-d2b1981d8fb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:45.363950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c784bb6-2d77-4d1b-8fd8-d2b1981d8fb8 None admin] RBAC: Adding query filter params (group.id=730ea508c3fd4575af2609bb7f189405, role.id=4552e69b96eb43f3ba6a9ca694d04ffb, scope.project.id=fe9516226a724cf688d3d23e1392161d) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:45.364472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c784bb6-2d77-4d1b-8fd8-d2b1981d8fb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:45.366366 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c784bb6-2d77-4d1b-8fd8-d2b1981d8fb8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:45.372811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 177/361] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jun 7 08:19:45 2022] GET /identity/v3/role_assignments?group.id=730ea508c3fd4575af2609bb7f189405&scope.project.id=fe9516226a724cf688d3d23e1392161d&role.id=4552e69b96eb43f3ba6a9ca694d04ffb => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:45.374511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c784bb6-2d77-4d1b-8fd8-d2b1981d8fb8 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:45.394575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:45.394767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 185/362] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:45.421930 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bbd20-5495-44d3-a5bb-c913302d755b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:45.490968 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c784bb6-2d77-4d1b-8fd8-d2b1981d8fb8 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:45.533566 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0002c-e7ad-4c31-b5ab-1d8c74806ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:45.534607 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0002c-e7ad-4c31-b5ab-1d8c74806ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:45.535084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0002c-e7ad-4c31-b5ab-1d8c74806ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:45.535551 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0002c-e7ad-4c31-b5ab-1d8c74806ca7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:45.544047 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bbd20-5495-44d3-a5bb-c913302d755b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:45.572459 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:45.586189 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b376e5-ddae-48a1-9a11-433e25b20ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:45.587245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b376e5-ddae-48a1-9a11-433e25b20ecb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:45.587535 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b376e5-ddae-48a1-9a11-433e25b20ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:45.588023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b376e5-ddae-48a1-9a11-433e25b20ecb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:45.603246 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b376e5-ddae-48a1-9a11-433e25b20ecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:45.612970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d6b376e5-ddae-48a1-9a11-433e25b20ecb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:19:45.612970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:45.612970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:45.612970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:45.612970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:45.612970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:45.612970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:45.612970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:45.612970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:45.612970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:45.612970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:45.612970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:45.612970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:45.612970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:45.612970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:45.612970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:45.613938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:45.613938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:45.613938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:45.613938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:45.613938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:45.613938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:45.613938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:45.613938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:45.613938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:45.613938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:45.613938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:45.613938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:45.613938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:45.613938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:45.613938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:45.613938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:45.614852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:45.614852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:45.614852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:45.614852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:45.614852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:45.614852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:45.614852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:45.614852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:45.614852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:45.614852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:45.614852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:19:45.614852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:45.619920 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 186/363] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:19:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:19:45.620636 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f2c6e-a24b-4540-b7a9-8383923e2c74 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:45.620927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f2c6e-a24b-4540-b7a9-8383923e2c74 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:45.621240 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f2c6e-a24b-4540-b7a9-8383923e2c74 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:45.622649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 187/364] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:45.626964 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0002c-e7ad-4c31-b5ab-1d8c74806ca7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:45.626964 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0002c-e7ad-4c31-b5ab-1d8c74806ca7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:45.628904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f2c6e-a24b-4540-b7a9-8383923e2c74 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:45.669372 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:45.747751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f2c6e-a24b-4540-b7a9-8383923e2c74 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:45.751738 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 178/365] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:45.753170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84526e-5a60-4173-ace2-3ef21b9a027c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:45.753550 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84526e-5a60-4173-ace2-3ef21b9a027c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:45.753920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84526e-5a60-4173-ace2-3ef21b9a027c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:45.772305 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ec84526e-5a60-4173-ace2-3ef21b9a027c None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:45.773682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ec84526e-5a60-4173-ace2-3ef21b9a027c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:45.786449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52411764-dc28-4ce6-8c65-cae67cc7c9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:45.787841 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52411764-dc28-4ce6-8c65-cae67cc7c9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:45.788336 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52411764-dc28-4ce6-8c65-cae67cc7c9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:45.788818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52411764-dc28-4ce6-8c65-cae67cc7c9d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:45.789890 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52411764-dc28-4ce6-8c65-cae67cc7c9d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:45.790408 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52411764-dc28-4ce6-8c65-cae67cc7c9d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:45.792033 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52411764-dc28-4ce6-8c65-cae67cc7c9d0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:45.802521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 188/366] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:45.803721 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52411764-dc28-4ce6-8c65-cae67cc7c9d0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:45.856101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 179/367] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:45.856955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84526e-5a60-4173-ace2-3ef21b9a027c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:45.865845 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:45.939407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:45.942751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52411764-dc28-4ce6-8c65-cae67cc7c9d0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:45.989070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3881d6a-b338-4539-a824-fe811de56690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:45.990418 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3881d6a-b338-4539-a824-fe811de56690 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:45.990904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3881d6a-b338-4539-a824-fe811de56690 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:45.991501 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3881d6a-b338-4539-a824-fe811de56690 None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:45.993609 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3881d6a-b338-4539-a824-fe811de56690 None admin] RBAC: Authorizing `identity:get_project(project_id=914fc5f0eaf745279782c417df7acf69)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:45.995589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3881d6a-b338-4539-a824-fe811de56690 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:45.998470 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 189/368] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:45 2022] GET /identity/v3/projects/914fc5f0eaf745279782c417df7acf69 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:45.999941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3881d6a-b338-4539-a824-fe811de56690 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:46.017443 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84526e-5a60-4173-ace2-3ef21b9a027c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:46.055723 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1bce1e-d02c-4ac8-ba2e-c28f8c0f0536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:46.056850 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bce1e-d02c-4ac8-ba2e-c28f8c0f0536 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:46.057361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bce1e-d02c-4ac8-ba2e-c28f8c0f0536 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:46.057827 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bce1e-d02c-4ac8-ba2e-c28f8c0f0536 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:46.114825 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3881d6a-b338-4539-a824-fe811de56690 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:46.130082 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:46.137057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1bce1e-d02c-4ac8-ba2e-c28f8c0f0536 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:46.138766 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1bce1e-d02c-4ac8-ba2e-c28f8c0f0536 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:46.151510 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82034a87-c361-4af4-9a5e-baa4990f31fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:46.152326 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82034a87-c361-4af4-9a5e-baa4990f31fa None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:46.152609 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82034a87-c361-4af4-9a5e-baa4990f31fa None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:46.152955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82034a87-c361-4af4-9a5e-baa4990f31fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:46.165889 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82034a87-c361-4af4-9a5e-baa4990f31fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:46.175792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-82034a87-c361-4af4-9a5e-baa4990f31fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:46.175792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:46.175792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:46.175792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:46.175792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:46.175792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:46.175792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:46.175792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:46.175792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:46.175792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:46.175792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:46.175792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:46.175792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:46.175792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:46.175792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:46.175792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:46.176724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:46.176724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:46.176724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:46.176724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:46.176724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:46.176724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:46.176724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:46.176724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:46.176724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:46.176724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:46.176724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:46.176724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:46.176724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:46.176724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:46.176724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:46.176724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:46.177541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:46.177541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:46.177541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:46.177541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:46.177541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:46.177541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:46.177541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:46.177541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:46.177541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:46.177541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:46.177541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:46.177541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:46.177541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 190/369] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:19:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:46.178174 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82034a87-c361-4af4-9a5e-baa4990f31fa None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:46.198851 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:46.258499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 180/370] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:46.259490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1bce1e-d02c-4ac8-ba2e-c28f8c0f0536 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:46.298794 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82034a87-c361-4af4-9a5e-baa4990f31fa None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:46.337066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eecd0f-d212-4d84-8ce7-83c71ad03a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:46.338174 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eecd0f-d212-4d84-8ce7-83c71ad03a62 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:46.338610 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eecd0f-d212-4d84-8ce7-83c71ad03a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:46.339041 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eecd0f-d212-4d84-8ce7-83c71ad03a62 None admin] PATH_INFO: `/v3/groups/730ea508c3fd4575af2609bb7f189405` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:46.340411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eecd0f-d212-4d84-8ce7-83c71ad03a62 None admin] RBAC: Authorizing `identity:get_group(group_id=730ea508c3fd4575af2609bb7f189405)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:46.342202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eecd0f-d212-4d84-8ce7-83c71ad03a62 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:46.344451 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 191/371] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jun 7 08:19:46 2022] GET /identity/v3/groups/730ea508c3fd4575af2609bb7f189405 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:46.349560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eecd0f-d212-4d84-8ce7-83c71ad03a62 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:46.368000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1bce1e-d02c-4ac8-ba2e-c28f8c0f0536 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:46.396893 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:46.407929 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98772c8-6b94-478d-8dee-65eb37bb3e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:46.409050 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98772c8-6b94-478d-8dee-65eb37bb3e82 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:46.409334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98772c8-6b94-478d-8dee-65eb37bb3e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:46.409564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98772c8-6b94-478d-8dee-65eb37bb3e82 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:46.410406 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98772c8-6b94-478d-8dee-65eb37bb3e82 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:46.410804 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98772c8-6b94-478d-8dee-65eb37bb3e82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:46.412261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98772c8-6b94-478d-8dee-65eb37bb3e82 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:46.420550 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 181/372] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:19:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:46.425812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98772c8-6b94-478d-8dee-65eb37bb3e82 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:46.438847 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:46.461130 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eecd0f-d212-4d84-8ce7-83c71ad03a62 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:46.497679 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16edd897-1979-4142-b497-75d13f4dcf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:46.499397 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edd897-1979-4142-b497-75d13f4dcf9d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:46.499949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edd897-1979-4142-b497-75d13f4dcf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:46.500432 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edd897-1979-4142-b497-75d13f4dcf9d None admin] PATH_INFO: `/v3/projects/df1e3ab9af2d4109a302f1229baa02a8` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:46.501835 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16edd897-1979-4142-b497-75d13f4dcf9d None admin] RBAC: Authorizing `identity:get_project(project_id=df1e3ab9af2d4109a302f1229baa02a8)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:46.503662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16edd897-1979-4142-b497-75d13f4dcf9d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:46.505564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 192/373] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:46 2022] GET /identity/v3/projects/df1e3ab9af2d4109a302f1229baa02a8 => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:46.506513 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16edd897-1979-4142-b497-75d13f4dcf9d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:46.536823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98772c8-6b94-478d-8dee-65eb37bb3e82 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:46.573738 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05c12d-d346-4130-aab1-0bace8cb72fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:46.574388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05c12d-d346-4130-aab1-0bace8cb72fd None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:46.574666 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05c12d-d346-4130-aab1-0bace8cb72fd None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:46.574935 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05c12d-d346-4130-aab1-0bace8cb72fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:46.575759 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05c12d-d346-4130-aab1-0bace8cb72fd None admin] RBAC: Adding query filter params (group.id=730ea508c3fd4575af2609bb7f189405, role.id=67111011e94c4e08928dd5e6b7a46af1, scope.project.id=914fc5f0eaf745279782c417df7acf69) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:46.576063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05c12d-d346-4130-aab1-0bace8cb72fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:46.578306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05c12d-d346-4130-aab1-0bace8cb72fd None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:46.584422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 182/374] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jun 7 08:19:46 2022] GET /identity/v3/role_assignments?group.id=730ea508c3fd4575af2609bb7f189405&scope.project.id=914fc5f0eaf745279782c417df7acf69&role.id=67111011e94c4e08928dd5e6b7a46af1 => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:46.585256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05c12d-d346-4130-aab1-0bace8cb72fd None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:46.618039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16edd897-1979-4142-b497-75d13f4dcf9d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:46.631456 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:46.661346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2768e0-0c06-4bff-acc0-d75cfa88005a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:46.662367 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2768e0-0c06-4bff-acc0-d75cfa88005a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:46.662878 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2768e0-0c06-4bff-acc0-d75cfa88005a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:46.663361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2768e0-0c06-4bff-acc0-d75cfa88005a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:46.691819 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:46.745420 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2768e0-0c06-4bff-acc0-d75cfa88005a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:46.746137 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05c12d-d346-4130-aab1-0bace8cb72fd None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:46.747552 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2768e0-0c06-4bff-acc0-d75cfa88005a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:46.803330 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e476ac4-5e0e-4228-a07e-043a562643ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:46.804213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e476ac4-5e0e-4228-a07e-043a562643ae None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:46.804492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e476ac4-5e0e-4228-a07e-043a562643ae None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:46.804768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e476ac4-5e0e-4228-a07e-043a562643ae None admin] PATH_INFO: `/v3/groups/250dc22123cd4c95a2884a5b83d0abd4` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:46.817916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e476ac4-5e0e-4228-a07e-043a562643ae None admin] RBAC: Authorizing `identity:get_group(group_id=250dc22123cd4c95a2884a5b83d0abd4)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:46.819466 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e476ac4-5e0e-4228-a07e-043a562643ae None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:46.821824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 183/375] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jun 7 08:19:46 2022] GET /identity/v3/groups/250dc22123cd4c95a2884a5b83d0abd4 => generated 229 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:46.827251 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e476ac4-5e0e-4228-a07e-043a562643ae None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:46.865620 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 193/376] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:46.871333 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2768e0-0c06-4bff-acc0-d75cfa88005a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:46.883907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:46.970805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e476ac4-5e0e-4228-a07e-043a562643ae None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:46.993441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2768e0-0c06-4bff-acc0-d75cfa88005a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:47.007710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582921d-4058-4c9a-a571-66f1c581f307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:47.008359 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582921d-4058-4c9a-a571-66f1c581f307 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:47.008636 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582921d-4058-4c9a-a571-66f1c581f307 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:47.008912 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582921d-4058-4c9a-a571-66f1c581f307 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:47.009769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6582921d-4058-4c9a-a571-66f1c581f307 None admin] RBAC: Adding query filter params (group.id=250dc22123cd4c95a2884a5b83d0abd4, role.id=8c63c00418a346b29f7db3f87a404e70, scope.project.id=df1e3ab9af2d4109a302f1229baa02a8) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:47.010077 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6582921d-4058-4c9a-a571-66f1c581f307 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:47.011582 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6582921d-4058-4c9a-a571-66f1c581f307 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:47.014590 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:47.018176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 184/377] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jun 7 08:19:46 2022] GET /identity/v3/role_assignments?group.id=250dc22123cd4c95a2884a5b83d0abd4&scope.project.id=df1e3ab9af2d4109a302f1229baa02a8&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:47.034630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e63e28-9595-4bd0-b021-fafedd04b5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:47.036058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e63e28-9595-4bd0-b021-fafedd04b5de None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:47.036544 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e63e28-9595-4bd0-b021-fafedd04b5de None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:47.037068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e63e28-9595-4bd0-b021-fafedd04b5de None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:47.038618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e63e28-9595-4bd0-b021-fafedd04b5de None admin] RBAC: Authorizing `identity:get_project(project_id=914fc5f0eaf745279782c417df7acf69)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:47.040342 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e63e28-9595-4bd0-b021-fafedd04b5de None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:47.043125 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 194/378] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:46 2022] GET /identity/v3/projects/914fc5f0eaf745279782c417df7acf69 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:47.049459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582921d-4058-4c9a-a571-66f1c581f307 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:47.189302 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582921d-4058-4c9a-a571-66f1c581f307 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:47.227214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ae860-01de-44f8-be8d-b7fc0853b2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:47.228084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ae860-01de-44f8-be8d-b7fc0853b2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:47.228421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ae860-01de-44f8-be8d-b7fc0853b2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:47.228696 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ae860-01de-44f8-be8d-b7fc0853b2a3 None admin] PATH_INFO: `/v3/groups/730ea508c3fd4575af2609bb7f189405` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:47.229932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ae860-01de-44f8-be8d-b7fc0853b2a3 None admin] RBAC: Authorizing `identity:get_group(group_id=730ea508c3fd4575af2609bb7f189405)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:47.231394 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ae860-01de-44f8-be8d-b7fc0853b2a3 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:47.233644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 185/379] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jun 7 08:19:47 2022] GET /identity/v3/groups/730ea508c3fd4575af2609bb7f189405 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:47.239764 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e63e28-9595-4bd0-b021-fafedd04b5de None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:47.344663 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:47.386872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e63e28-9595-4bd0-b021-fafedd04b5de None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:47.425515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4603e-b3b6-48b4-94b3-f0647741e503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:47.426176 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4603e-b3b6-48b4-94b3-f0647741e503 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:47.426458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4603e-b3b6-48b4-94b3-f0647741e503 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:47.426733 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4603e-b3b6-48b4-94b3-f0647741e503 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:47.427630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4603e-b3b6-48b4-94b3-f0647741e503 None admin] RBAC: Adding query filter params (group.id=730ea508c3fd4575af2609bb7f189405, role.id=4552e69b96eb43f3ba6a9ca694d04ffb, scope.project.id=914fc5f0eaf745279782c417df7acf69) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:47.427940 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4603e-b3b6-48b4-94b3-f0647741e503 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:47.429456 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4603e-b3b6-48b4-94b3-f0647741e503 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:47.435616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 195/380] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jun 7 08:19:47 2022] GET /identity/v3/role_assignments?group.id=730ea508c3fd4575af2609bb7f189405&scope.project.id=914fc5f0eaf745279782c417df7acf69&role.id=4552e69b96eb43f3ba6a9ca694d04ffb => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:47.745137 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b87670-7e06-4f08-947c-7893b1308374 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:47.745137 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b87670-7e06-4f08-947c-7893b1308374 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:47.745137 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b87670-7e06-4f08-947c-7893b1308374 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:47.745916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 186/381] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:47.751575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f91a7-d8b2-4a14-89a5-9c3882406b1d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:47.752461 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f91a7-d8b2-4a14-89a5-9c3882406b1d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:47.752938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f91a7-d8b2-4a14-89a5-9c3882406b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:47.771457 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ff2f91a7-d8b2-4a14-89a5-9c3882406b1d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:47.773027 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ff2f91a7-d8b2-4a14-89a5-9c3882406b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:47.848246 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 196/382] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:47.868200 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b87670-7e06-4f08-947c-7893b1308374 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:47.951195 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:47.990676 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b87670-7e06-4f08-947c-7893b1308374 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:48.036464 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995fe8d-81b2-42d9-87f3-e0c7fa1e7abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:48.037217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995fe8d-81b2-42d9-87f3-e0c7fa1e7abf None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:48.037498 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995fe8d-81b2-42d9-87f3-e0c7fa1e7abf None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:48.037771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995fe8d-81b2-42d9-87f3-e0c7fa1e7abf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:48.055145 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995fe8d-81b2-42d9-87f3-e0c7fa1e7abf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:48.063165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-7995fe8d-81b2-42d9-87f3-e0c7fa1e7abf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:48.063165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:48.063165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:48.063165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:48.063165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:48.063165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:48.063165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:48.063165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:48.063165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:48.063165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:48.063165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:48.063165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:48.063165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:48.063165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:48.063165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:48.063165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:48.064100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:48.064100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:48.064100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:48.064100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:48.064100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:48.064100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:48.064100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:48.064100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:48.064100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:48.064100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:48.064100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:48.064100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:48.064100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:48.064100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:48.064100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:48.064100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:48.064884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:48.064884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:48.064884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:48.064884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:48.064884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:48.064884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:48.064884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:48.064884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:48.064884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:48.064884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:48.064884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:48.064884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:48.064884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 187/383] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:48.070756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f91a7-d8b2-4a14-89a5-9c3882406b1d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:48.148782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:48.184027 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f91a7-d8b2-4a14-89a5-9c3882406b1d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:48.221367 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fed27f-f3f7-4357-b2ee-da5337d76527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:48.222051 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fed27f-f3f7-4357-b2ee-da5337d76527 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:48.222339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fed27f-f3f7-4357-b2ee-da5337d76527 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:48.222621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fed27f-f3f7-4357-b2ee-da5337d76527 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:48.223372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fed27f-f3f7-4357-b2ee-da5337d76527 None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:48.223690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fed27f-f3f7-4357-b2ee-da5337d76527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:48.225101 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fed27f-f3f7-4357-b2ee-da5337d76527 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:48.234085 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 197/384] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:48.239731 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995fe8d-81b2-42d9-87f3-e0c7fa1e7abf None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:48.302551 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf57d3-6390-447b-8da3-d1c3ace57563 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:48.302740 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf57d3-6390-447b-8da3-d1c3ace57563 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:48.303011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf57d3-6390-447b-8da3-d1c3ace57563 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:48.304497 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 198/385] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:19:48.309789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3fe8a-9e72-42f8-8339-558469089464 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:48.310070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3fe8a-9e72-42f8-8339-558469089464 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:48.310346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3fe8a-9e72-42f8-8339-558469089464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:48.335710 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1da3fe8a-9e72-42f8-8339-558469089464 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:48.337953 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1da3fe8a-9e72-42f8-8339-558469089464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:48.406303 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995fe8d-81b2-42d9-87f3-e0c7fa1e7abf None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:48.455780 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:48.461053 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 199/386] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:48.470674 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23566076-46d3-4156-9ea5-6a18e2d0ba37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:48.471541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23566076-46d3-4156-9ea5-6a18e2d0ba37 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:48.471819 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23566076-46d3-4156-9ea5-6a18e2d0ba37 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:48.475536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23566076-46d3-4156-9ea5-6a18e2d0ba37 None admin] PATH_INFO: `/v3/groups/730ea508c3fd4575af2609bb7f189405` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:48.475536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23566076-46d3-4156-9ea5-6a18e2d0ba37 None admin] RBAC: Authorizing `identity:get_group(group_id=730ea508c3fd4575af2609bb7f189405)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:48.475536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23566076-46d3-4156-9ea5-6a18e2d0ba37 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:48.482399 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3fe8a-9e72-42f8-8339-558469089464 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:48.487985 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 188/387] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jun 7 08:19:48 2022] GET /identity/v3/groups/730ea508c3fd4575af2609bb7f189405 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:48.489257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23566076-46d3-4156-9ea5-6a18e2d0ba37 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:48.585129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:48.601813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23566076-46d3-4156-9ea5-6a18e2d0ba37 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:48.641855 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac5c56-5759-439f-b75d-f0dc0b48981d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:48.643033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac5c56-5759-439f-b75d-f0dc0b48981d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:48.643565 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac5c56-5759-439f-b75d-f0dc0b48981d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:48.644061 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac5c56-5759-439f-b75d-f0dc0b48981d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:48.655456 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac5c56-5759-439f-b75d-f0dc0b48981d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:48.664232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-f0ac5c56-5759-439f-b75d-f0dc0b48981d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:48.664232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:48.664232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:48.664232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:48.664232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:48.664232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:48.664232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:48.664232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:48.664232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:48.664232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:48.664232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:48.664232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:48.664232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:48.664232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:48.664232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:48.664232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:48.665160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:48.665160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:48.665160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:48.665160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:48.665160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:48.665160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:48.665160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:48.665160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:48.665160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:48.665160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:48.665160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:48.665160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:48.665160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:48.665160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:48.665160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:48.665160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:48.665919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:48.665919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:48.665919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:48.665919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:48.665919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:48.665919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:48.665919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:48.665919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:48.665919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:48.665919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:48.665919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:48.665919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:48.667659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3fe8a-9e72-42f8-8339-558469089464 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:48.668782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 189/388] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:19:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:48.675170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac5c56-5759-439f-b75d-f0dc0b48981d None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:48.741448 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:48.760353 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f559bc-9671-48d9-82b8-1c2224bba42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:48.761263 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f559bc-9671-48d9-82b8-1c2224bba42f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:48.761547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f559bc-9671-48d9-82b8-1c2224bba42f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:48.761820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f559bc-9671-48d9-82b8-1c2224bba42f None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:48.763131 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f559bc-9671-48d9-82b8-1c2224bba42f None admin] RBAC: Authorizing `identity:get_project(project_id=fe9516226a724cf688d3d23e1392161d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:48.765071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f559bc-9671-48d9-82b8-1c2224bba42f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:48.767052 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 200/389] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:48 2022] GET /identity/v3/projects/fe9516226a724cf688d3d23e1392161d => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:48.778005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f559bc-9671-48d9-82b8-1c2224bba42f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:48.787896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac5c56-5759-439f-b75d-f0dc0b48981d None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:48.825647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548953fc-9058-4779-a791-4fe9c60b73c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:48.827309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548953fc-9058-4779-a791-4fe9c60b73c9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:48.827833 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548953fc-9058-4779-a791-4fe9c60b73c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:48.828301 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548953fc-9058-4779-a791-4fe9c60b73c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:48.829449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548953fc-9058-4779-a791-4fe9c60b73c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:48.830357 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548953fc-9058-4779-a791-4fe9c60b73c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:48.832118 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548953fc-9058-4779-a791-4fe9c60b73c9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:48.841909 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 190/390] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:19:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:48.847994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548953fc-9058-4779-a791-4fe9c60b73c9 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:48.960348 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548953fc-9058-4779-a791-4fe9c60b73c9 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:48.979947 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:49.000733 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a12cc-d4cd-4be5-ac01-eb4169a1fb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:49.004836 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a12cc-d4cd-4be5-ac01-eb4169a1fb73 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:49.004836 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a12cc-d4cd-4be5-ac01-eb4169a1fb73 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:49.004836 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a12cc-d4cd-4be5-ac01-eb4169a1fb73 None admin] PATH_INFO: `/v3/groups/730ea508c3fd4575af2609bb7f189405` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:49.004836 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a12cc-d4cd-4be5-ac01-eb4169a1fb73 None admin] RBAC: Authorizing `identity:get_group(group_id=730ea508c3fd4575af2609bb7f189405)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:49.006005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a12cc-d4cd-4be5-ac01-eb4169a1fb73 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:49.008321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 191/391] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jun 7 08:19:48 2022] GET /identity/v3/groups/730ea508c3fd4575af2609bb7f189405 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:49.015484 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a12cc-d4cd-4be5-ac01-eb4169a1fb73 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:49.126846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a12cc-d4cd-4be5-ac01-eb4169a1fb73 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:49.134936 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f559bc-9671-48d9-82b8-1c2224bba42f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:49.163845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da481546-7d94-48fd-b04d-445be3c83715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:49.165084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da481546-7d94-48fd-b04d-445be3c83715 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:49.165561 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da481546-7d94-48fd-b04d-445be3c83715 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:49.166440 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da481546-7d94-48fd-b04d-445be3c83715 None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:49.167853 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da481546-7d94-48fd-b04d-445be3c83715 None admin] RBAC: Authorizing `identity:get_project(project_id=fe9516226a724cf688d3d23e1392161d)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:49.169635 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da481546-7d94-48fd-b04d-445be3c83715 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:49.172355 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 192/392] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:49 2022] GET /identity/v3/projects/fe9516226a724cf688d3d23e1392161d => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:49.180088 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da481546-7d94-48fd-b04d-445be3c83715 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:49.211907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:49.221056 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6bee03-3dcb-4440-a807-9ec2e788596d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:49.221426 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6bee03-3dcb-4440-a807-9ec2e788596d None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:49.230658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6bee03-3dcb-4440-a807-9ec2e788596d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:49.230658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6bee03-3dcb-4440-a807-9ec2e788596d None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d/groups/730ea508c3fd4575af2609bb7f189405/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:49.230658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6bee03-3dcb-4440-a807-9ec2e788596d None admin] RBAC: Authorizing `identity:create_grant(project_id=fe9516226a724cf688d3d23e1392161d, group_id=730ea508c3fd4575af2609bb7f189405, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:49.230658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6bee03-3dcb-4440-a807-9ec2e788596d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:49.265407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:49.310807 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 201/393] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Jun 7 08:19:48 2022] PUT /identity/v3/projects/fe9516226a724cf688d3d23e1392161d/groups/730ea508c3fd4575af2609bb7f189405/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:19:49.312165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da481546-7d94-48fd-b04d-445be3c83715 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:49.359972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97e10e-4a85-4754-a28c-0c199f95c1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:49.360946 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97e10e-4a85-4754-a28c-0c199f95c1ae None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:49.361220 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97e10e-4a85-4754-a28c-0c199f95c1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:49.361556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97e10e-4a85-4754-a28c-0c199f95c1ae None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d/groups/730ea508c3fd4575af2609bb7f189405/roles/4552e69b96eb43f3ba6a9ca694d04ffb` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:49.363984 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97e10e-4a85-4754-a28c-0c199f95c1ae None admin] RBAC: Authorizing `identity:create_grant(project_id=fe9516226a724cf688d3d23e1392161d, group_id=730ea508c3fd4575af2609bb7f189405, role_id=4552e69b96eb43f3ba6a9ca694d04ffb)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:49.365533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97e10e-4a85-4754-a28c-0c199f95c1ae None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:49.403246 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def7ccb-81d0-47e1-badb-dc406d534cbc None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:49.403524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def7ccb-81d0-47e1-badb-dc406d534cbc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:49.403741 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def7ccb-81d0-47e1-badb-dc406d534cbc None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:49.405897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 202/394] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:49.412195 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435da6d-ffb6-4bfe-accd-b9efa3f96826 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:49.412798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435da6d-ffb6-4bfe-accd-b9efa3f96826 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:49.413304 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435da6d-ffb6-4bfe-accd-b9efa3f96826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:49.433977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d435da6d-ffb6-4bfe-accd-b9efa3f96826 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:49.435597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d435da6d-ffb6-4bfe-accd-b9efa3f96826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:49.448430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 193/395] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Jun 7 08:19:49 2022] PUT /identity/v3/projects/fe9516226a724cf688d3d23e1392161d/groups/730ea508c3fd4575af2609bb7f189405/roles/4552e69b96eb43f3ba6a9ca694d04ffb => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:19:49.525548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 203/396] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:49.565118 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97e10e-4a85-4754-a28c-0c199f95c1ae None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:49.697776 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97e10e-4a85-4754-a28c-0c199f95c1ae None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:49.715424 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:49.739504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13085a2b-a996-4779-8c16-ce78efc43518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:49.740316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13085a2b-a996-4779-8c16-ce78efc43518 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:49.740599 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13085a2b-a996-4779-8c16-ce78efc43518 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:49.740874 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13085a2b-a996-4779-8c16-ce78efc43518 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:49.757337 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13085a2b-a996-4779-8c16-ce78efc43518 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:49.770057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-13085a2b-a996-4779-8c16-ce78efc43518 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:49.770057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:49.770057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:49.770057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:49.770057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:49.770057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:49.770057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:49.770057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:49.770057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:49.770057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:49.770057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:49.770057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:49.770057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:49.770057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:49.770057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:49.770057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:49.770999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:49.770999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:49.770999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:49.770999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:49.770999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:49.770999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:49.770999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:49.770999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:49.770999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:49.770999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:49.770999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:49.770999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:49.770999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:49.770999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:49.770999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:49.770999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:49.771805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:49.771805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:49.771805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:49.771805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:49.771805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:49.771805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:49.771805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:49.771805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:49.771805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:49.771805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:49.771805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:49.771805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:49.771805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 194/397] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:49.778776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435da6d-ffb6-4bfe-accd-b9efa3f96826 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:49.921377 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435da6d-ffb6-4bfe-accd-b9efa3f96826 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:49.953248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea6fab-f43f-4cd7-af57-b33fe34eeec5 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:49.953248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea6fab-f43f-4cd7-af57-b33fe34eeec5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:49.953248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea6fab-f43f-4cd7-af57-b33fe34eeec5 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:49.954187 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 195/398] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:49.962040 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac64c9-b019-437e-bc74-06f01d5c07ea None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:49.962658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:49.963478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac64c9-b019-437e-bc74-06f01d5c07ea None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:49.963988 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac64c9-b019-437e-bc74-06f01d5c07ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:49.985614 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-6fac64c9-b019-437e-bc74-06f01d5c07ea None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:49.987176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6fac64c9-b019-437e-bc74-06f01d5c07ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:49.991011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958e82c-a40e-4022-b066-375b5113b9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:49.991011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958e82c-a40e-4022-b066-375b5113b9ac None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:49.991351 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958e82c-a40e-4022-b066-375b5113b9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:49.991607 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958e82c-a40e-4022-b066-375b5113b9ac None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:49.992565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958e82c-a40e-4022-b066-375b5113b9ac None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:49.992877 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958e82c-a40e-4022-b066-375b5113b9ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:49.994293 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958e82c-a40e-4022-b066-375b5113b9ac None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:50.013245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 204/399] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:50.030946 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958e82c-a40e-4022-b066-375b5113b9ac None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:50.073155 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 196/400] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:50.098942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac64c9-b019-437e-bc74-06f01d5c07ea None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:50.226038 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac64c9-b019-437e-bc74-06f01d5c07ea None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:50.229384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:50.268765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a742ab-5408-422a-adf6-e3b13756e771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:50.269788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a742ab-5408-422a-adf6-e3b13756e771 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:50.270081 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a742ab-5408-422a-adf6-e3b13756e771 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:50.270365 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a742ab-5408-422a-adf6-e3b13756e771 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:50.284763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a742ab-5408-422a-adf6-e3b13756e771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:50.293209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-60a742ab-5408-422a-adf6-e3b13756e771 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:19:50.293209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:50.293209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:50.293209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:50.293209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:50.293209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:50.293209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:50.293209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:50.293209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:50.293209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:50.293209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:50.293209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:50.293209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:50.293209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:50.293209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:50.293209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:50.294165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:50.294165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:50.294165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:50.294165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:50.294165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:50.294165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:50.294165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:50.294165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:50.294165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:50.294165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:50.294165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:50.294165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:50.294165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:50.294165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:50.294165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:50.294165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:50.295478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:50.295478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:50.295478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:50.295478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:50.295478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:50.295478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:50.295478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:50.295478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:50.295478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:50.295478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:50.295478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:19:50.295478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:50.295478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 197/401] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:19:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:19:50.301848 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a742ab-5408-422a-adf6-e3b13756e771 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:50.339478 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958e82c-a40e-4022-b066-375b5113b9ac None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:50.416629 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a742ab-5408-422a-adf6-e3b13756e771 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:50.456124 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7942553-24af-4fd8-8e28-9cbf092144a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:50.456798 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7942553-24af-4fd8-8e28-9cbf092144a7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:50.457139 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7942553-24af-4fd8-8e28-9cbf092144a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:50.457425 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7942553-24af-4fd8-8e28-9cbf092144a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:50.458189 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7942553-24af-4fd8-8e28-9cbf092144a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:50.458289 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57743029-2479-47fb-9f61-7b8f1b025c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:50.458494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7942553-24af-4fd8-8e28-9cbf092144a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:50.459104 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57743029-2479-47fb-9f61-7b8f1b025c26 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:50.459388 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57743029-2479-47fb-9f61-7b8f1b025c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:50.459661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57743029-2479-47fb-9f61-7b8f1b025c26 None admin] PATH_INFO: `/v3/groups/730ea508c3fd4575af2609bb7f189405` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:50.460385 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7942553-24af-4fd8-8e28-9cbf092144a7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:50.460828 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57743029-2479-47fb-9f61-7b8f1b025c26 None admin] RBAC: Authorizing `identity:get_group(group_id=730ea508c3fd4575af2609bb7f189405)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:50.462269 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57743029-2479-47fb-9f61-7b8f1b025c26 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:50.463885 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 205/402] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jun 7 08:19:50 2022] GET /identity/v3/groups/730ea508c3fd4575af2609bb7f189405 => generated 226 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:50.472641 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57743029-2479-47fb-9f61-7b8f1b025c26 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:50.480832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 198/403] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:50.492441 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7942553-24af-4fd8-8e28-9cbf092144a7 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:50.573422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:50.605026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:50.705803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7942553-24af-4fd8-8e28-9cbf092144a7 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:50.705997 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57743029-2479-47fb-9f61-7b8f1b025c26 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:50.787583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b7e7cd-1e10-48b5-a1be-9805fa8ed979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:50.787786 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33145b91-4a69-4f99-82f0-50b8dc5c79ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:50.793025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7e7cd-1e10-48b5-a1be-9805fa8ed979 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:50.793141 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33145b91-4a69-4f99-82f0-50b8dc5c79ac None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:50.793216 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7e7cd-1e10-48b5-a1be-9805fa8ed979 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:50.793284 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33145b91-4a69-4f99-82f0-50b8dc5c79ac None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:50.793376 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7e7cd-1e10-48b5-a1be-9805fa8ed979 None admin] PATH_INFO: `/v3/groups/250dc22123cd4c95a2884a5b83d0abd4` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:50.793446 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33145b91-4a69-4f99-82f0-50b8dc5c79ac None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:50.793795 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b7e7cd-1e10-48b5-a1be-9805fa8ed979 None admin] RBAC: Authorizing `identity:get_group(group_id=250dc22123cd4c95a2884a5b83d0abd4)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:50.794120 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33145b91-4a69-4f99-82f0-50b8dc5c79ac None admin] RBAC: Authorizing `identity:get_project(project_id=914fc5f0eaf745279782c417df7acf69)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:50.795292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b7e7cd-1e10-48b5-a1be-9805fa8ed979 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:50.795622 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33145b91-4a69-4f99-82f0-50b8dc5c79ac None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:50.797437 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 199/404] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jun 7 08:19:50 2022] GET /identity/v3/groups/250dc22123cd4c95a2884a5b83d0abd4 => generated 229 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:50.801321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 206/405] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:50 2022] GET /identity/v3/projects/914fc5f0eaf745279782c417df7acf69 => generated 302 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:50.814117 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b7e7cd-1e10-48b5-a1be-9805fa8ed979 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:50.817826 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33145b91-4a69-4f99-82f0-50b8dc5c79ac None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:51.024109 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33145b91-4a69-4f99-82f0-50b8dc5c79ac None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:51.025352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b7e7cd-1e10-48b5-a1be-9805fa8ed979 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:51.041439 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:51.079613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:51.082069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27a1e6-421c-4d4a-a420-ed78490cce51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:51.083191 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27a1e6-421c-4d4a-a420-ed78490cce51 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:51.083472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27a1e6-421c-4d4a-a420-ed78490cce51 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:51.083740 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27a1e6-421c-4d4a-a420-ed78490cce51 None admin] PATH_INFO: `/v3/projects/df1e3ab9af2d4109a302f1229baa02a8` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:51.085328 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27a1e6-421c-4d4a-a420-ed78490cce51 None admin] RBAC: Authorizing `identity:get_project(project_id=df1e3ab9af2d4109a302f1229baa02a8)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:51.086808 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27a1e6-421c-4d4a-a420-ed78490cce51 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:51.089101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 200/406] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:50 2022] GET /identity/v3/projects/df1e3ab9af2d4109a302f1229baa02a8 => generated 344 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:51.090680 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abb2a3-1da4-40dc-a36c-35ca642ac455 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:51.090967 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abb2a3-1da4-40dc-a36c-35ca642ac455 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:51.091246 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abb2a3-1da4-40dc-a36c-35ca642ac455 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:51.094254 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 201/407] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:51.100204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fa2cc-1bd1-483e-b568-670b13b8666c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:51.102197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fa2cc-1bd1-483e-b568-670b13b8666c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:51.102197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fa2cc-1bd1-483e-b568-670b13b8666c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:51.122599 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-0d2fa2cc-1bd1-483e-b568-670b13b8666c None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:51.124397 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0d2fa2cc-1bd1-483e-b568-670b13b8666c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:51.127118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3292670-f8c7-4683-81d8-3e5648fcd442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:51.128193 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3292670-f8c7-4683-81d8-3e5648fcd442 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:51.128476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3292670-f8c7-4683-81d8-3e5648fcd442 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:51.128753 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3292670-f8c7-4683-81d8-3e5648fcd442 None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69/groups/730ea508c3fd4575af2609bb7f189405/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:51.134686 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3292670-f8c7-4683-81d8-3e5648fcd442 None admin] RBAC: Authorizing `identity:create_grant(project_id=914fc5f0eaf745279782c417df7acf69, group_id=730ea508c3fd4575af2609bb7f189405, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:51.138372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3292670-f8c7-4683-81d8-3e5648fcd442 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:51.224833 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 207/408] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Jun 7 08:19:50 2022] PUT /identity/v3/projects/914fc5f0eaf745279782c417df7acf69/groups/730ea508c3fd4575af2609bb7f189405/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:19:51.225441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3292670-f8c7-4683-81d8-3e5648fcd442 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:51.244471 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 202/409] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:51.316075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fa2cc-1bd1-483e-b568-670b13b8666c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:51.408147 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3292670-f8c7-4683-81d8-3e5648fcd442 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:51.476102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:51.483876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27f22e-3419-45d8-a7fe-f6bfe0d2be56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:51.484976 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27f22e-3419-45d8-a7fe-f6bfe0d2be56 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:51.485430 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27f22e-3419-45d8-a7fe-f6bfe0d2be56 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:51.491478 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27f22e-3419-45d8-a7fe-f6bfe0d2be56 None admin] PATH_INFO: `/v3/projects/df1e3ab9af2d4109a302f1229baa02a8/groups/250dc22123cd4c95a2884a5b83d0abd4/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:51.491596 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fa2cc-1bd1-483e-b568-670b13b8666c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:51.491676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27f22e-3419-45d8-a7fe-f6bfe0d2be56 None admin] RBAC: Authorizing `identity:create_grant(project_id=df1e3ab9af2d4109a302f1229baa02a8, group_id=250dc22123cd4c95a2884a5b83d0abd4, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:51.492648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27f22e-3419-45d8-a7fe-f6bfe0d2be56 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:51.534179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460956b9-7cdb-4a3e-ba56-2fd90f21fce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:51.535318 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460956b9-7cdb-4a3e-ba56-2fd90f21fce6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:51.535746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460956b9-7cdb-4a3e-ba56-2fd90f21fce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:51.536164 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460956b9-7cdb-4a3e-ba56-2fd90f21fce6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:51.550077 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460956b9-7cdb-4a3e-ba56-2fd90f21fce6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:51.558769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-460956b9-7cdb-4a3e-ba56-2fd90f21fce6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:51.558769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:51.558769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:51.558769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:51.558769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:51.558769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:51.558769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:51.558769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:51.558769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:51.558769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:51.558769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:51.558769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:51.558769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:51.558769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:51.558769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:51.558769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:51.559815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:51.559815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:51.559815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:51.559815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:51.559815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:51.559815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:51.559815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:51.559815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:51.559815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:51.559815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:51.559815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:51.559815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:51.559815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:51.559815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:51.559815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:51.559815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:51.561388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:51.561388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:51.561388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:51.561388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:51.561388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:51.561388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:51.561388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:51.561388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:51.561388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:51.561388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:51.561388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:51.561388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:51.563767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 203/410] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:19:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:51.574060 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460956b9-7cdb-4a3e-ba56-2fd90f21fce6 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:51.574869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 208/411] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Jun 7 08:19:51 2022] PUT /identity/v3/projects/df1e3ab9af2d4109a302f1229baa02a8/groups/250dc22123cd4c95a2884a5b83d0abd4/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:19:51.720112 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460956b9-7cdb-4a3e-ba56-2fd90f21fce6 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:51.771493 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275dd3b3-f323-48a1-838b-6fbd6950494a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:51.772172 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dd3b3-f323-48a1-838b-6fbd6950494a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:51.772452 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dd3b3-f323-48a1-838b-6fbd6950494a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:51.772728 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dd3b3-f323-48a1-838b-6fbd6950494a None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:51.773534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275dd3b3-f323-48a1-838b-6fbd6950494a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:51.773843 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275dd3b3-f323-48a1-838b-6fbd6950494a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:51.775255 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275dd3b3-f323-48a1-838b-6fbd6950494a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:51.789418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 204/412] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:19:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:51.795212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27f22e-3419-45d8-a7fe-f6bfe0d2be56 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:51.844916 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:51.909244 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27f22e-3419-45d8-a7fe-f6bfe0d2be56 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:51.950746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bae3b3-bcdf-4dc1-a6ec-c68bd3eb01aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:51.951698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bae3b3-bcdf-4dc1-a6ec-c68bd3eb01aa None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:51.951976 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bae3b3-bcdf-4dc1-a6ec-c68bd3eb01aa None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:51.952245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bae3b3-bcdf-4dc1-a6ec-c68bd3eb01aa None admin] PATH_INFO: `/v3/groups/730ea508c3fd4575af2609bb7f189405` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:51.953571 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bae3b3-bcdf-4dc1-a6ec-c68bd3eb01aa None admin] RBAC: Authorizing `identity:get_group(group_id=730ea508c3fd4575af2609bb7f189405)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:51.955089 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bae3b3-bcdf-4dc1-a6ec-c68bd3eb01aa None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:51.957030 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 209/413] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jun 7 08:19:51 2022] GET /identity/v3/groups/730ea508c3fd4575af2609bb7f189405 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:51.962314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275dd3b3-f323-48a1-838b-6fbd6950494a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:51.997169 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:52.129780 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275dd3b3-f323-48a1-838b-6fbd6950494a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:52.189027 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49bb04-4489-4845-8ae7-c94af52a39cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:52.189027 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49bb04-4489-4845-8ae7-c94af52a39cd None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:52.189227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49bb04-4489-4845-8ae7-c94af52a39cd None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:52.189486 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49bb04-4489-4845-8ae7-c94af52a39cd None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:52.193768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49bb04-4489-4845-8ae7-c94af52a39cd None admin] RBAC: Authorizing `identity:get_project(project_id=914fc5f0eaf745279782c417df7acf69)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:52.195218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49bb04-4489-4845-8ae7-c94af52a39cd None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:52.197296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 205/414] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:51 2022] GET /identity/v3/projects/914fc5f0eaf745279782c417df7acf69 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:52.202577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bae3b3-bcdf-4dc1-a6ec-c68bd3eb01aa None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:52.355324 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bae3b3-bcdf-4dc1-a6ec-c68bd3eb01aa None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:52.364817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:52.389062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdfc8e-3236-4b4e-9b6a-20cfbc74889f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:52.389941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdfc8e-3236-4b4e-9b6a-20cfbc74889f None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:52.390222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdfc8e-3236-4b4e-9b6a-20cfbc74889f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:52.390493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdfc8e-3236-4b4e-9b6a-20cfbc74889f None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69/groups/730ea508c3fd4575af2609bb7f189405/roles/4552e69b96eb43f3ba6a9ca694d04ffb` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:52.392728 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdfc8e-3236-4b4e-9b6a-20cfbc74889f None admin] RBAC: Authorizing `identity:create_grant(project_id=914fc5f0eaf745279782c417df7acf69, group_id=730ea508c3fd4575af2609bb7f189405, role_id=4552e69b96eb43f3ba6a9ca694d04ffb)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:52.394187 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdfc8e-3236-4b4e-9b6a-20cfbc74889f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:52.434744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdbb46-7c1c-4c8e-b186-cea7ebc9612c None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:52.435084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdbb46-7c1c-4c8e-b186-cea7ebc9612c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:52.435304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdbb46-7c1c-4c8e-b186-cea7ebc9612c None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:52.436935 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 206/415] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:52.441915 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a0995-1028-47a5-b0d6-1f64d7f32bd7 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:52.442301 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a0995-1028-47a5-b0d6-1f64d7f32bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:52.442685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a0995-1028-47a5-b0d6-1f64d7f32bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:52.459885 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-de6a0995-1028-47a5-b0d6-1f64d7f32bd7 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:52.461070 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-de6a0995-1028-47a5-b0d6-1f64d7f32bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:52.499462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 210/416] 10.222.0.17 () {58 vars in 1442 bytes} [Tue Jun 7 08:19:52 2022] PUT /identity/v3/projects/914fc5f0eaf745279782c417df7acf69/groups/730ea508c3fd4575af2609bb7f189405/roles/4552e69b96eb43f3ba6a9ca694d04ffb => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:19:52.500836 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241865f2-b451-437e-bf6e-b98962c73e91 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:52.501389 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241865f2-b451-437e-bf6e-b98962c73e91 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:52.501841 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241865f2-b451-437e-bf6e-b98962c73e91 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:52.503690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 211/417] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:52.508909 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ffeb4-9a81-461b-8926-182bb4e39756 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:52.509451 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ffeb4-9a81-461b-8926-182bb4e39756 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:52.510011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ffeb4-9a81-461b-8926-182bb4e39756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:52.532863 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 207/418] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:52.540154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9d9ffeb4-9a81-461b-8926-182bb4e39756 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:52.541366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9d9ffeb4-9a81-461b-8926-182bb4e39756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:52.573758 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a0995-1028-47a5-b0d6-1f64d7f32bd7 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:52.597363 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:52.666846 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 212/419] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:52.702906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ffeb4-9a81-461b-8926-182bb4e39756 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:52.753318 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a0995-1028-47a5-b0d6-1f64d7f32bd7 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:52.790637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8074a0-89fb-4505-aacf-2d1f37ec5a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:52.791475 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8074a0-89fb-4505-aacf-2d1f37ec5a97 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:52.791754 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8074a0-89fb-4505-aacf-2d1f37ec5a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:52.792023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8074a0-89fb-4505-aacf-2d1f37ec5a97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:52.804474 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8074a0-89fb-4505-aacf-2d1f37ec5a97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:52.812857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-9b8074a0-89fb-4505-aacf-2d1f37ec5a97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:52.812857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:52.812857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:52.812857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:52.812857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:52.812857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:52.812857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:52.812857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:52.812857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:52.812857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:52.812857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:52.812857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:52.812857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:52.812857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:52.812857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:52.812857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:52.813868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:52.813868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:52.813868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:52.813868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:52.813868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:52.813868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:52.813868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:52.813868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:52.813868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:52.813868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:52.813868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:52.813868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:52.813868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:52.813868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:52.813868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:52.813868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:52.814654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:52.814654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:52.814654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:52.814654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:52.814654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:52.814654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:52.814654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:52.814654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:52.814654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:52.814654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:52.814654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:52.814654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:52.814654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 208/420] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:52.822203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8074a0-89fb-4505-aacf-2d1f37ec5a97 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:52.846792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ffeb4-9a81-461b-8926-182bb4e39756 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:52.855557 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:52.894443 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:52.902522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f071b-cad2-4138-b03f-3862f89c7a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:52.903557 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f071b-cad2-4138-b03f-3862f89c7a3f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:52.904017 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f071b-cad2-4138-b03f-3862f89c7a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:52.905032 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f071b-cad2-4138-b03f-3862f89c7a3f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:52.925634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8074a0-89fb-4505-aacf-2d1f37ec5a97 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:52.926264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f071b-cad2-4138-b03f-3862f89c7a3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:52.934561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-745f071b-cad2-4138-b03f-3862f89c7a3f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:52.934561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:52.934561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:52.934561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:52.934561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:52.934561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:52.934561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:52.934561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:52.934561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:52.934561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:52.934561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:52.934561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:52.934561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:52.934561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:52.934561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:52.934561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:52.935449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:52.935449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:52.935449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:52.935449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:52.935449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:52.935449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:52.935449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:52.935449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:52.935449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:52.935449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:52.935449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:52.935449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:52.935449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:52.935449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:52.935449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:52.935449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:52.936247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:52.936247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:52.936247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:52.936247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:52.936247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:52.936247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:52.936247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:52.936247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:52.936247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:52.936247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:52.936247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:52.936247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:19:52.946948 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 213/421] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:19:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:52.952911 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f071b-cad2-4138-b03f-3862f89c7a3f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:52.959449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c946c7-4d6c-4982-ac2b-f1c3746a43bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:52.960114 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c946c7-4d6c-4982-ac2b-f1c3746a43bd None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:52.960393 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c946c7-4d6c-4982-ac2b-f1c3746a43bd None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:52.960670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c946c7-4d6c-4982-ac2b-f1c3746a43bd None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:52.961507 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c946c7-4d6c-4982-ac2b-f1c3746a43bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:52.961812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c946c7-4d6c-4982-ac2b-f1c3746a43bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:52.969021 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c946c7-4d6c-4982-ac2b-f1c3746a43bd None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:52.972508 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 209/422] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:52.978138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c946c7-4d6c-4982-ac2b-f1c3746a43bd None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:53.084774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c946c7-4d6c-4982-ac2b-f1c3746a43bd None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:53.147470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c3372-f16c-43fa-80e0-7c3a8a92bde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:53.148125 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c3372-f16c-43fa-80e0-7c3a8a92bde4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:53.148375 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c3372-f16c-43fa-80e0-7c3a8a92bde4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:53.149791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c3372-f16c-43fa-80e0-7c3a8a92bde4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:53.153341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:53.165277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f071b-cad2-4138-b03f-3862f89c7a3f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:53.224685 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566af376-bc9a-43db-88e1-273d943fc35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:53.225868 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566af376-bc9a-43db-88e1-273d943fc35a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:53.226372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566af376-bc9a-43db-88e1-273d943fc35a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:53.226979 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566af376-bc9a-43db-88e1-273d943fc35a None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:53.227927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566af376-bc9a-43db-88e1-273d943fc35a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:53.228449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566af376-bc9a-43db-88e1-273d943fc35a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:53.230288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566af376-bc9a-43db-88e1-273d943fc35a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:53.246243 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c3372-f16c-43fa-80e0-7c3a8a92bde4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:53.248032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c3372-f16c-43fa-80e0-7c3a8a92bde4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:53.252425 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 214/423] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:19:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:53.266248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566af376-bc9a-43db-88e1-273d943fc35a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:53.316191 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:53.400840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 210/424] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 08:19:53.404145 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c3372-f16c-43fa-80e0-7c3a8a92bde4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:53.457995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:53.470334 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566af376-bc9a-43db-88e1-273d943fc35a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:53.511115 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9c256-ffb9-4831-9b2e-9a25fdb46045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:53.511770 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9c256-ffb9-4831-9b2e-9a25fdb46045 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:53.512048 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9c256-ffb9-4831-9b2e-9a25fdb46045 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:53.512320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9c256-ffb9-4831-9b2e-9a25fdb46045 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:53.599732 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c3372-f16c-43fa-80e0-7c3a8a92bde4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:53.642631 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08806efd-415f-4466-8ad0-012d166663fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:53.643732 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08806efd-415f-4466-8ad0-012d166663fa None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:53.644210 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08806efd-415f-4466-8ad0-012d166663fa None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:53.644618 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08806efd-415f-4466-8ad0-012d166663fa None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:53.645981 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08806efd-415f-4466-8ad0-012d166663fa None admin] RBAC: Authorizing `identity:get_project(project_id=fe9516226a724cf688d3d23e1392161d)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:53.647789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08806efd-415f-4466-8ad0-012d166663fa None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:53.650352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 211/425] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:53 2022] GET /identity/v3/projects/fe9516226a724cf688d3d23e1392161d => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:53.656697 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08806efd-415f-4466-8ad0-012d166663fa None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:53.682039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9c256-ffb9-4831-9b2e-9a25fdb46045 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:53.683515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9c256-ffb9-4831-9b2e-9a25fdb46045 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:53.725428 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:53.770786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08806efd-415f-4466-8ad0-012d166663fa None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:53.772715 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:53.814462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f855a1b-9be5-4223-b102-7050b5fb7413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:53.815325 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f855a1b-9be5-4223-b102-7050b5fb7413 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:53.815604 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f855a1b-9be5-4223-b102-7050b5fb7413 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:53.815875 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f855a1b-9be5-4223-b102-7050b5fb7413 None admin] PATH_INFO: `/v3/groups/730ea508c3fd4575af2609bb7f189405` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:53.819092 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f855a1b-9be5-4223-b102-7050b5fb7413 None admin] RBAC: Authorizing `identity:get_group(group_id=730ea508c3fd4575af2609bb7f189405)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:53.820545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f855a1b-9be5-4223-b102-7050b5fb7413 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:53.832179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 212/426] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jun 7 08:19:53 2022] GET /identity/v3/groups/730ea508c3fd4575af2609bb7f189405 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:53.846411 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f855a1b-9be5-4223-b102-7050b5fb7413 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:53.932103 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 215/427] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 08:19:53.938865 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9c256-ffb9-4831-9b2e-9a25fdb46045 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:53.964124 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f855a1b-9be5-4223-b102-7050b5fb7413 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:54.011809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f40b7f-d1b0-4872-a9c7-1829ff5bb6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:54.012786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f40b7f-d1b0-4872-a9c7-1829ff5bb6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:54.013306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f40b7f-d1b0-4872-a9c7-1829ff5bb6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:54.014389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f40b7f-d1b0-4872-a9c7-1829ff5bb6c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:54.015578 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f40b7f-d1b0-4872-a9c7-1829ff5bb6c5 None admin] RBAC: Adding query filter params (group.id=730ea508c3fd4575af2609bb7f189405, role.id=67111011e94c4e08928dd5e6b7a46af1, scope.project.id=fe9516226a724cf688d3d23e1392161d) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:54.016108 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f40b7f-d1b0-4872-a9c7-1829ff5bb6c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:54.017953 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f40b7f-d1b0-4872-a9c7-1829ff5bb6c5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:54.024805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 213/428] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jun 7 08:19:53 2022] GET /identity/v3/role_assignments?group.id=730ea508c3fd4575af2609bb7f189405&scope.project.id=fe9516226a724cf688d3d23e1392161d&role.id=67111011e94c4e08928dd5e6b7a46af1 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:54.026573 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d51684-eb87-4f85-8746-13f68d103e8d None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:54.027083 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d51684-eb87-4f85-8746-13f68d103e8d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:54.027565 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d51684-eb87-4f85-8746-13f68d103e8d None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:54.033617 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 214/429] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:54.035747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baaf66-221f-4af3-a33a-c719e6f8022e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:54.036360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baaf66-221f-4af3-a33a-c719e6f8022e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:54.036850 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baaf66-221f-4af3-a33a-c719e6f8022e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:54.065771 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9c256-ffb9-4831-9b2e-9a25fdb46045 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:54.067144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-76baaf66-221f-4af3-a33a-c719e6f8022e None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:54.068791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-76baaf66-221f-4af3-a33a-c719e6f8022e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:54.089650 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:54.121783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449ecb7-b1c6-47c0-a9aa-3fe38fceadfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:54.123049 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449ecb7-b1c6-47c0-a9aa-3fe38fceadfc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:54.124123 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449ecb7-b1c6-47c0-a9aa-3fe38fceadfc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:54.124597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449ecb7-b1c6-47c0-a9aa-3fe38fceadfc None admin] PATH_INFO: `/v3/projects/fe9516226a724cf688d3d23e1392161d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:54.126184 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449ecb7-b1c6-47c0-a9aa-3fe38fceadfc None admin] RBAC: Authorizing `identity:get_project(project_id=fe9516226a724cf688d3d23e1392161d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:54.127018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:54.128533 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449ecb7-b1c6-47c0-a9aa-3fe38fceadfc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:54.135147 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 216/430] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:53 2022] GET /identity/v3/projects/fe9516226a724cf688d3d23e1392161d => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:54.136776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449ecb7-b1c6-47c0-a9aa-3fe38fceadfc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:54.221433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 215/431] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:54.256111 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76baaf66-221f-4af3-a33a-c719e6f8022e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:54.296452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449ecb7-b1c6-47c0-a9aa-3fe38fceadfc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:54.339433 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2845196-d5d1-4030-8215-639fda1e90d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:54.340292 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2845196-d5d1-4030-8215-639fda1e90d0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:54.340573 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2845196-d5d1-4030-8215-639fda1e90d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:54.340848 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2845196-d5d1-4030-8215-639fda1e90d0 None admin] PATH_INFO: `/v3/groups/730ea508c3fd4575af2609bb7f189405` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:54.342659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2845196-d5d1-4030-8215-639fda1e90d0 None admin] RBAC: Authorizing `identity:get_group(group_id=730ea508c3fd4575af2609bb7f189405)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:54.344096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2845196-d5d1-4030-8215-639fda1e90d0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:54.347421 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 217/432] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jun 7 08:19:54 2022] GET /identity/v3/groups/730ea508c3fd4575af2609bb7f189405 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:54.347421 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681f063-86ec-41d9-b57a-8ee9060eef3c None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:54.347421 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681f063-86ec-41d9-b57a-8ee9060eef3c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:54.347697 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681f063-86ec-41d9-b57a-8ee9060eef3c None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:54.349139 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 218/433] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:54.351426 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681f063-86ec-41d9-b57a-8ee9060eef3c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:54.400248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76baaf66-221f-4af3-a33a-c719e6f8022e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:54.425875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:54.437964 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44cbbcd-22bf-43b2-b470-a2f5bfe9460c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:54.441623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44cbbcd-22bf-43b2-b470-a2f5bfe9460c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:54.441623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44cbbcd-22bf-43b2-b470-a2f5bfe9460c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:54.441623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44cbbcd-22bf-43b2-b470-a2f5bfe9460c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:54.466770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44cbbcd-22bf-43b2-b470-a2f5bfe9460c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:54.473470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-f44cbbcd-22bf-43b2-b470-a2f5bfe9460c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:54.473470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:54.473470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:54.473470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:54.473470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:54.473470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:54.473470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:54.473470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:54.473470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:54.473470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:54.473470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:54.473470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:54.473470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:54.473470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:54.473470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:54.473470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:54.474542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:54.474542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:54.474542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:54.474542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:54.474542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:54.474542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:54.474542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:54.474542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:54.474542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:54.474542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:54.474542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:54.474542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:54.474542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:54.474542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:54.474542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:54.474542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:54.475628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:54.475628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:54.475628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:54.475628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:54.475628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:54.475628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:54.475628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:54.475628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:54.475628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:54.475628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:54.475628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 08:19:54.475628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:54.475628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 216/434] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:19:54.476409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f0141-3694-4cf4-9511-aa67f6c72dfd None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:54.476474 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f0141-3694-4cf4-9511-aa67f6c72dfd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:54.476713 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f0141-3694-4cf4-9511-aa67f6c72dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:54.488401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681f063-86ec-41d9-b57a-8ee9060eef3c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:54.500499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-968f0141-3694-4cf4-9511-aa67f6c72dfd None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:54.501767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-968f0141-3694-4cf4-9511-aa67f6c72dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:54.522932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:54.536878 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74753711-afc0-4877-b515-5ee855830145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:54.538015 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74753711-afc0-4877-b515-5ee855830145 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:54.538451 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74753711-afc0-4877-b515-5ee855830145 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:54.540119 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74753711-afc0-4877-b515-5ee855830145 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:54.540119 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74753711-afc0-4877-b515-5ee855830145 None admin] RBAC: Adding query filter params (group.id=730ea508c3fd4575af2609bb7f189405, role.id=4552e69b96eb43f3ba6a9ca694d04ffb, scope.project.id=fe9516226a724cf688d3d23e1392161d) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:54.540314 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74753711-afc0-4877-b515-5ee855830145 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:54.542464 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74753711-afc0-4877-b515-5ee855830145 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:54.548782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 219/435] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jun 7 08:19:54 2022] GET /identity/v3/role_assignments?group.id=730ea508c3fd4575af2609bb7f189405&scope.project.id=fe9516226a724cf688d3d23e1392161d&role.id=4552e69b96eb43f3ba6a9ca694d04ffb => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:54.550855 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74753711-afc0-4877-b515-5ee855830145 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:54.586471 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 217/436] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:54.618644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f0141-3694-4cf4-9511-aa67f6c72dfd None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:54.693534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74753711-afc0-4877-b515-5ee855830145 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:54.716704 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:54.733625 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a00352-47c4-462c-bc2e-a4a378057940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:54.734728 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a00352-47c4-462c-bc2e-a4a378057940 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:54.735708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a00352-47c4-462c-bc2e-a4a378057940 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:54.735708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a00352-47c4-462c-bc2e-a4a378057940 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:54.736775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a00352-47c4-462c-bc2e-a4a378057940 None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:54.737316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a00352-47c4-462c-bc2e-a4a378057940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:54.739096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a00352-47c4-462c-bc2e-a4a378057940 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:54.751587 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 220/437] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:19:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:54.757042 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f0141-3694-4cf4-9511-aa67f6c72dfd None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:54.758294 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a00352-47c4-462c-bc2e-a4a378057940 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:54.798712 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135a1d7-411a-413e-ae54-d06d46686395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:54.800226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135a1d7-411a-413e-ae54-d06d46686395 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:54.800226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135a1d7-411a-413e-ae54-d06d46686395 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:54.800441 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135a1d7-411a-413e-ae54-d06d46686395 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:54.813029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135a1d7-411a-413e-ae54-d06d46686395 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:54.821552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-6135a1d7-411a-413e-ae54-d06d46686395 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:19:54.821552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:54.821552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:54.821552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:54.821552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:54.821552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:54.821552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:54.821552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:54.821552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:54.821552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:54.821552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:54.821552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:54.821552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:54.821552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:54.821552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:54.821552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:54.822436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:54.822436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:54.822436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:54.822436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:54.822436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:54.822436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:54.822436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:54.822436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:54.822436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:54.822436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:54.822436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:54.822436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:54.822436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:54.822436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:54.822436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:54.822436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:54.823202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:54.823202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:54.823202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:54.823202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:54.823202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:54.823202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:54.823202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:54.823202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:54.823202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:54.823202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:54.823202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:19:54.823202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:54.823202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 218/438] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:19:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:19:54.829940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135a1d7-411a-413e-ae54-d06d46686395 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:54.850504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:54.896627 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a00352-47c4-462c-bc2e-a4a378057940 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:54.939882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b78d7-49db-4bbc-9c9b-08db933b17a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:54.941389 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b78d7-49db-4bbc-9c9b-08db933b17a9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:54.941867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b78d7-49db-4bbc-9c9b-08db933b17a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:54.942346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b78d7-49db-4bbc-9c9b-08db933b17a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:54.973609 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135a1d7-411a-413e-ae54-d06d46686395 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:55.002844 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:55.020968 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87b2fa-0aa7-4a56-968d-ee67610866f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:55.021670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87b2fa-0aa7-4a56-968d-ee67610866f6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:55.021954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87b2fa-0aa7-4a56-968d-ee67610866f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:55.022226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87b2fa-0aa7-4a56-968d-ee67610866f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:55.022972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87b2fa-0aa7-4a56-968d-ee67610866f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:55.023275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87b2fa-0aa7-4a56-968d-ee67610866f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:55.024644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87b2fa-0aa7-4a56-968d-ee67610866f6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:55.032179 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b78d7-49db-4bbc-9c9b-08db933b17a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:55.034078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b78d7-49db-4bbc-9c9b-08db933b17a9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:55.035288 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 219/439] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:19:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:55.044389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87b2fa-0aa7-4a56-968d-ee67610866f6 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:55.144626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:55.157142 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 221/440] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:55.162735 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b78d7-49db-4bbc-9c9b-08db933b17a9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:55.167922 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87b2fa-0aa7-4a56-968d-ee67610866f6 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:55.208697 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03aca1-d0de-4fd0-94f7-6092bade0996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:55.209372 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03aca1-d0de-4fd0-94f7-6092bade0996 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:55.209654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03aca1-d0de-4fd0-94f7-6092bade0996 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:55.209929 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03aca1-d0de-4fd0-94f7-6092bade0996 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:55.245856 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:55.263476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b78d7-49db-4bbc-9c9b-08db933b17a9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:55.277301 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03aca1-d0de-4fd0-94f7-6092bade0996 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:55.278775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03aca1-d0de-4fd0-94f7-6092bade0996 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:55.297443 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e5228-c884-4757-b9fa-3e59c3499e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:55.298302 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e5228-c884-4757-b9fa-3e59c3499e5f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:55.298588 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e5228-c884-4757-b9fa-3e59c3499e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:55.298914 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e5228-c884-4757-b9fa-3e59c3499e5f None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:55.300354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e5228-c884-4757-b9fa-3e59c3499e5f None admin] RBAC: Authorizing `identity:get_project(project_id=914fc5f0eaf745279782c417df7acf69)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:55.301909 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e5228-c884-4757-b9fa-3e59c3499e5f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:55.303597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 222/441] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:55 2022] GET /identity/v3/projects/914fc5f0eaf745279782c417df7acf69 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:55.304754 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53da8a2-1bdd-45e5-95ef-b276275fac22 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:55.305057 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53da8a2-1bdd-45e5-95ef-b276275fac22 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:55.305341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53da8a2-1bdd-45e5-95ef-b276275fac22 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:55.306784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 223/442] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:55.308980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53da8a2-1bdd-45e5-95ef-b276275fac22 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:55.411649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53da8a2-1bdd-45e5-95ef-b276275fac22 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:55.447814 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56c2dd-f509-4826-99cb-c48c6b1bfa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:55.448690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56c2dd-f509-4826-99cb-c48c6b1bfa46 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:55.448967 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56c2dd-f509-4826-99cb-c48c6b1bfa46 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:55.449267 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56c2dd-f509-4826-99cb-c48c6b1bfa46 None admin] PATH_INFO: `/v3/groups/730ea508c3fd4575af2609bb7f189405` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:55.450500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56c2dd-f509-4826-99cb-c48c6b1bfa46 None admin] RBAC: Authorizing `identity:get_group(group_id=730ea508c3fd4575af2609bb7f189405)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:55.452622 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56c2dd-f509-4826-99cb-c48c6b1bfa46 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:55.454562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 224/443] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jun 7 08:19:55 2022] GET /identity/v3/groups/730ea508c3fd4575af2609bb7f189405 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:55.456665 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8b770-0ed8-4f33-9040-f492703e2c1f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:55.456950 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8b770-0ed8-4f33-9040-f492703e2c1f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:55.457254 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8b770-0ed8-4f33-9040-f492703e2c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:55.474040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-10f8b770-0ed8-4f33-9040-f492703e2c1f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:55.475234 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-10f8b770-0ed8-4f33-9040-f492703e2c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:55.475903 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:55.488181 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:55.499337 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 220/444] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:19:55.500158 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03aca1-d0de-4fd0-94f7-6092bade0996 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:55.541318 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 225/445] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:55.542148 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8b770-0ed8-4f33-9040-f492703e2c1f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:55.604502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03aca1-d0de-4fd0-94f7-6092bade0996 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:55.638331 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0651187-75a2-4c03-90dd-a0f0eff650e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:55.639117 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0651187-75a2-4c03-90dd-a0f0eff650e3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:55.639390 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0651187-75a2-4c03-90dd-a0f0eff650e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:55.639661 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0651187-75a2-4c03-90dd-a0f0eff650e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:55.640472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0651187-75a2-4c03-90dd-a0f0eff650e3 None admin] RBAC: Adding query filter params (group.id=730ea508c3fd4575af2609bb7f189405, role.id=67111011e94c4e08928dd5e6b7a46af1, scope.project.id=914fc5f0eaf745279782c417df7acf69) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:55.640776 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0651187-75a2-4c03-90dd-a0f0eff650e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:55.642224 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0651187-75a2-4c03-90dd-a0f0eff650e3 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:55.647701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8b770-0ed8-4f33-9040-f492703e2c1f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:55.647949 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 221/446] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jun 7 08:19:55 2022] GET /identity/v3/role_assignments?group.id=730ea508c3fd4575af2609bb7f189405&scope.project.id=914fc5f0eaf745279782c417df7acf69&role.id=67111011e94c4e08928dd5e6b7a46af1 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:55.648835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0651187-75a2-4c03-90dd-a0f0eff650e3 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:55.690625 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742435c4-a9e8-4828-9c5b-626fde553f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:55.691514 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742435c4-a9e8-4828-9c5b-626fde553f5a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:55.691802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742435c4-a9e8-4828-9c5b-626fde553f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:55.692080 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742435c4-a9e8-4828-9c5b-626fde553f5a None admin] PATH_INFO: `/v3/projects/df1e3ab9af2d4109a302f1229baa02a8` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:55.693373 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742435c4-a9e8-4828-9c5b-626fde553f5a None admin] RBAC: Authorizing `identity:get_project(project_id=df1e3ab9af2d4109a302f1229baa02a8)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:55.694845 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742435c4-a9e8-4828-9c5b-626fde553f5a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:55.696761 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 226/447] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:55 2022] GET /identity/v3/projects/df1e3ab9af2d4109a302f1229baa02a8 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:55.698120 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:55.702150 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742435c4-a9e8-4828-9c5b-626fde553f5a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:55.721955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:55.763915 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0651187-75a2-4c03-90dd-a0f0eff650e3 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:55.804398 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f776c06-1cbd-46be-b585-d85166e3b25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:55.805241 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f776c06-1cbd-46be-b585-d85166e3b25d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:55.805544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f776c06-1cbd-46be-b585-d85166e3b25d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:55.805797 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f776c06-1cbd-46be-b585-d85166e3b25d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:55.812304 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742435c4-a9e8-4828-9c5b-626fde553f5a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:55.818776 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f776c06-1cbd-46be-b585-d85166e3b25d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:19:55.828291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-2f776c06-1cbd-46be-b585-d85166e3b25d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:55.828291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:19:55.828291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:19:55.828291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:19:55.828291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:19:55.828291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:19:55.828291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:19:55.828291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:19:55.828291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:19:55.828291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:19:55.828291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:19:55.828291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:19:55.828291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:19:55.828291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:19:55.828291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:19:55.828291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:19:55.829261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:19:55.829261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:19:55.829261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:19:55.829261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:19:55.829261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:19:55.829261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:19:55.829261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:19:55.829261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:19:55.829261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:19:55.829261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:19:55.829261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:19:55.829261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:19:55.829261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:19:55.829261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:19:55.829261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:19:55.829261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:19:55.830173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:19:55.830173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:19:55.830173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:19:55.830173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:19:55.830173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:19:55.830173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:19:55.830173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:19:55.830173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:19:55.830173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:19:55.830173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:19:55.830173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:19:55.830173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:19:55.830173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 222/448] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:19:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:19:55.835271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f776c06-1cbd-46be-b585-d85166e3b25d None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:55.855342 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db9d86-fd46-45c3-89cc-db6d53aca366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:55.856222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db9d86-fd46-45c3-89cc-db6d53aca366 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:55.856502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db9d86-fd46-45c3-89cc-db6d53aca366 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:55.856775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db9d86-fd46-45c3-89cc-db6d53aca366 None admin] PATH_INFO: `/v3/groups/250dc22123cd4c95a2884a5b83d0abd4` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:55.860283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db9d86-fd46-45c3-89cc-db6d53aca366 None admin] RBAC: Authorizing `identity:get_group(group_id=250dc22123cd4c95a2884a5b83d0abd4)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:55.860283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db9d86-fd46-45c3-89cc-db6d53aca366 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:55.861855 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 227/449] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jun 7 08:19:55 2022] GET /identity/v3/groups/250dc22123cd4c95a2884a5b83d0abd4 => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:55.867503 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db9d86-fd46-45c3-89cc-db6d53aca366 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:55.953222 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:55.954889 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f776c06-1cbd-46be-b585-d85166e3b25d None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:55.961133 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:55.979273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db9d86-fd46-45c3-89cc-db6d53aca366 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:55.995398 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e207b4-2a24-4d01-8ee9-2f9d42148553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:55.996075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e207b4-2a24-4d01-8ee9-2f9d42148553 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:55.996359 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e207b4-2a24-4d01-8ee9-2f9d42148553 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:55.996629 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e207b4-2a24-4d01-8ee9-2f9d42148553 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:55.997418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e207b4-2a24-4d01-8ee9-2f9d42148553 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:56.000475 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e207b4-2a24-4d01-8ee9-2f9d42148553 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:56.000475 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e207b4-2a24-4d01-8ee9-2f9d42148553 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:56.008036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 223/450] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:19:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:56.013823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e207b4-2a24-4d01-8ee9-2f9d42148553 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:56.018509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5226ab7-5d18-460a-97d2-b615ddec4a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:56.019206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5226ab7-5d18-460a-97d2-b615ddec4a52 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:56.019452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5226ab7-5d18-460a-97d2-b615ddec4a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:56.019750 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5226ab7-5d18-460a-97d2-b615ddec4a52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:56.020585 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5226ab7-5d18-460a-97d2-b615ddec4a52 None admin] RBAC: Adding query filter params (group.id=250dc22123cd4c95a2884a5b83d0abd4, role.id=8c63c00418a346b29f7db3f87a404e70, scope.project.id=df1e3ab9af2d4109a302f1229baa02a8) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:56.020888 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5226ab7-5d18-460a-97d2-b615ddec4a52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:56.022342 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5226ab7-5d18-460a-97d2-b615ddec4a52 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:56.028632 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 228/451] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jun 7 08:19:55 2022] GET /identity/v3/role_assignments?group.id=250dc22123cd4c95a2884a5b83d0abd4&scope.project.id=df1e3ab9af2d4109a302f1229baa02a8&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:19:56.126590 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e207b4-2a24-4d01-8ee9-2f9d42148553 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:56.161193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2983a7-5c34-4856-8c9d-33c005a2f9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:56.161857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2983a7-5c34-4856-8c9d-33c005a2f9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:56.162144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2983a7-5c34-4856-8c9d-33c005a2f9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:56.162415 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2983a7-5c34-4856-8c9d-33c005a2f9b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:56.198817 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:56.238933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2983a7-5c34-4856-8c9d-33c005a2f9b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:56.240315 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2983a7-5c34-4856-8c9d-33c005a2f9b1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:56.336251 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 224/452] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:19:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 08:19:56.342423 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5226ab7-5d18-460a-97d2-b615ddec4a52 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:56.437204 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5226ab7-5d18-460a-97d2-b615ddec4a52 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:56.468959 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da20340-d69b-4d62-bdc3-d5da6d5f9e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:56.469821 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da20340-d69b-4d62-bdc3-d5da6d5f9e21 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:56.470098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da20340-d69b-4d62-bdc3-d5da6d5f9e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:56.470369 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da20340-d69b-4d62-bdc3-d5da6d5f9e21 None admin] PATH_INFO: `/v3/projects/914fc5f0eaf745279782c417df7acf69` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:56.471511 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da20340-d69b-4d62-bdc3-d5da6d5f9e21 None admin] RBAC: Authorizing `identity:get_project(project_id=914fc5f0eaf745279782c417df7acf69)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:56.472938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da20340-d69b-4d62-bdc3-d5da6d5f9e21 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:56.474583 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 229/453] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:19:56 2022] GET /identity/v3/projects/914fc5f0eaf745279782c417df7acf69 => generated 302 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:56.479791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2983a7-5c34-4856-8c9d-33c005a2f9b1 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:56.538163 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:56.575078 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2983a7-5c34-4856-8c9d-33c005a2f9b1 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:56.607698 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd60e83-2e58-4a62-9899-0cf78775fa31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:56.608533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd60e83-2e58-4a62-9899-0cf78775fa31 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:56.608816 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd60e83-2e58-4a62-9899-0cf78775fa31 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:56.609118 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd60e83-2e58-4a62-9899-0cf78775fa31 None admin] PATH_INFO: `/v3/groups/730ea508c3fd4575af2609bb7f189405` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:56.610272 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd60e83-2e58-4a62-9899-0cf78775fa31 None admin] RBAC: Authorizing `identity:get_group(group_id=730ea508c3fd4575af2609bb7f189405)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:56.611736 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd60e83-2e58-4a62-9899-0cf78775fa31 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:56.613338 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 225/454] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Jun 7 08:19:56 2022] GET /identity/v3/groups/730ea508c3fd4575af2609bb7f189405 => generated 226 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:56.618870 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da20340-d69b-4d62-bdc3-d5da6d5f9e21 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:56.636594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:56.714826 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da20340-d69b-4d62-bdc3-d5da6d5f9e21 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:56.746707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb77778-bacd-43db-aa83-c12d822477e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:19:56.747377 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb77778-bacd-43db-aa83-c12d822477e9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:56.747659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb77778-bacd-43db-aa83-c12d822477e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:56.747938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb77778-bacd-43db-aa83-c12d822477e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:56.749314 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb77778-bacd-43db-aa83-c12d822477e9 None admin] RBAC: Adding query filter params (group.id=730ea508c3fd4575af2609bb7f189405, role.id=4552e69b96eb43f3ba6a9ca694d04ffb, scope.project.id=914fc5f0eaf745279782c417df7acf69) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:19:56.749659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb77778-bacd-43db-aa83-c12d822477e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:19:56.751042 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb77778-bacd-43db-aa83-c12d822477e9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:19:56.756572 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 230/455] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Jun 7 08:19:56 2022] GET /identity/v3/role_assignments?group.id=730ea508c3fd4575af2609bb7f189405&scope.project.id=914fc5f0eaf745279782c417df7acf69&role.id=4552e69b96eb43f3ba6a9ca694d04ffb => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:19:59.550764 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d01b9e-9fda-4c86-a169-5c8c861c8e47 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:59.551245 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d01b9e-9fda-4c86-a169-5c8c861c8e47 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:59.551245 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d01b9e-9fda-4c86-a169-5c8c861c8e47 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:59.552875 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 226/456] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:59.558079 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a283aa-38a0-4330-ac0d-256de56e4516 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:59.558881 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a283aa-38a0-4330-ac0d-256de56e4516 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:59.559351 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a283aa-38a0-4330-ac0d-256de56e4516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:59.586012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-87a283aa-38a0-4330-ac0d-256de56e4516 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:59.587558 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-87a283aa-38a0-4330-ac0d-256de56e4516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:59.600410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fad4e-ee62-4575-8479-fbd97fdfe953 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:59.601021 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fad4e-ee62-4575-8479-fbd97fdfe953 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:59.601499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fad4e-ee62-4575-8479-fbd97fdfe953 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:59.603060 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 227/457] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:19:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:19:59.611533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb161b64-ff0e-4056-88ec-62c8e5a1cca2 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:19:59.612052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb161b64-ff0e-4056-88ec-62c8e5a1cca2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:19:59.612580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb161b64-ff0e-4056-88ec-62c8e5a1cca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:19:59.646022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-cb161b64-ff0e-4056-88ec-62c8e5a1cca2 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:19:59.647344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cb161b64-ff0e-4056-88ec-62c8e5a1cca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:19:59.669745 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 231/458] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:19:59.718201 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a283aa-38a0-4330-ac0d-256de56e4516 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:59.736700 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:59.764320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 228/459] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:19:59.791651 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb161b64-ff0e-4056-88ec-62c8e5a1cca2 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:19:59.909882 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:19:59.932274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a283aa-38a0-4330-ac0d-256de56e4516 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:19:59.987582 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb161b64-ff0e-4056-88ec-62c8e5a1cca2 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:00.004784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e2bc5-3736-4671-a83d-ebae408b6d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:00.005826 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e2bc5-3736-4671-a83d-ebae408b6d61 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:00.006255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e2bc5-3736-4671-a83d-ebae408b6d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:00.006672 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e2bc5-3736-4671-a83d-ebae408b6d61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:00.017834 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e2bc5-3736-4671-a83d-ebae408b6d61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:00.019523 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e2bc5-3736-4671-a83d-ebae408b6d61 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:00.038540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-114e2bc5-3736-4671-a83d-ebae408b6d61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:00.038540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:00.038540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:00.038540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:00.038540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:00.038540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:00.038540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:00.038540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:00.038540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:00.038540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:00.038540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:00.038540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:00.038540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:00.038540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:00.038540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:00.039616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:00.039616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:00.039616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:00.039616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:00.039616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:00.039616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:00.039616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:00.039616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:00.039616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:00.039616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:00.039616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:00.039616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:00.039616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:00.039616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:00.039616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:00.039616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:00.039616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:00.040650 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:00.040650 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:00.040650 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:00.040650 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:00.042524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 232/460] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:00.048797 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e2bc5-3736-4671-a83d-ebae408b6d61 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:00.065694 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9cee0-f97d-4a17-b370-47e3a168e18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:00.066762 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9cee0-f97d-4a17-b370-47e3a168e18e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:00.067188 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9cee0-f97d-4a17-b370-47e3a168e18e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:00.067618 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9cee0-f97d-4a17-b370-47e3a168e18e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:00.091042 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b9cee0-f97d-4a17-b370-47e3a168e18e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:00.093350 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b9cee0-f97d-4a17-b370-47e3a168e18e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:00.114147 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:00.124920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-66b9cee0-f97d-4a17-b370-47e3a168e18e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:00.124920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:00.124920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:00.124920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:00.124920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:00.124920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:00.124920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:00.124920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:00.124920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:00.124920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:00.124920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:00.124920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:00.124920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:00.124920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:00.124920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:00.125999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:00.125999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:00.125999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:00.125999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:00.125999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:00.125999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:00.125999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:00.125999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:00.125999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:00.125999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:00.125999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:00.125999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:00.125999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:00.125999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:00.125999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:00.125999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:00.125999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:00.127022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:00.127022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:00.127022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:00.127022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:00.134461 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 229/461] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:00.143093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9cee0-f97d-4a17-b370-47e3a168e18e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:00.159372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e2bc5-3736-4671-a83d-ebae408b6d61 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:00.194688 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a279e7e-aefd-4439-bbab-e36428029e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:00.195356 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a279e7e-aefd-4439-bbab-e36428029e59 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:00.195646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a279e7e-aefd-4439-bbab-e36428029e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:00.195928 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a279e7e-aefd-4439-bbab-e36428029e59 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:00.196657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a279e7e-aefd-4439-bbab-e36428029e59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:00.196968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a279e7e-aefd-4439-bbab-e36428029e59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:00.198578 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a279e7e-aefd-4439-bbab-e36428029e59 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:00.218482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 233/462] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:00.223901 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a279e7e-aefd-4439-bbab-e36428029e59 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:00.293265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9cee0-f97d-4a17-b370-47e3a168e18e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:00.308833 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:00.329257 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a279e7e-aefd-4439-bbab-e36428029e59 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:00.333459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d3528-de5a-43c9-a74f-21f497f1d85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:00.334119 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d3528-de5a-43c9-a74f-21f497f1d85e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:00.334398 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d3528-de5a-43c9-a74f-21f497f1d85e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:00.334666 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d3528-de5a-43c9-a74f-21f497f1d85e None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:00.335386 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d3528-de5a-43c9-a74f-21f497f1d85e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:00.335689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d3528-de5a-43c9-a74f-21f497f1d85e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:00.337093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d3528-de5a-43c9-a74f-21f497f1d85e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:00.350080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 230/463] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:00.354461 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:00.355814 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb95b4-e7f8-47d1-a2aa-708cabd6da9a None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:00.355927 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb95b4-e7f8-47d1-a2aa-708cabd6da9a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:00.356182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb95b4-e7f8-47d1-a2aa-708cabd6da9a None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:00.361871 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 231/464] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:00.361871 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd5671-947c-4614-a52e-c5c9f7d2e0b9 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:00.361871 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd5671-947c-4614-a52e-c5c9f7d2e0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:00.361871 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd5671-947c-4614-a52e-c5c9f7d2e0b9 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:00.361871 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 232/465] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:00.361871 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd5671-947c-4614-a52e-c5c9f7d2e0b9 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:00.373298 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35daf70b-b36f-4685-816f-f0fa4a17c38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:00.373969 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35daf70b-b36f-4685-816f-f0fa4a17c38b None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:00.374255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35daf70b-b36f-4685-816f-f0fa4a17c38b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:00.374538 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35daf70b-b36f-4685-816f-f0fa4a17c38b None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:00.375265 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35daf70b-b36f-4685-816f-f0fa4a17c38b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:00.376713 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35daf70b-b36f-4685-816f-f0fa4a17c38b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:00.464569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd5671-947c-4614-a52e-c5c9f7d2e0b9 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:00.482547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 234/466] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jun 7 08:20:00 2022] POST /identity/v3/users => generated 246 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:20:00.484183 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030939c-44fe-4f6f-9c99-cb9b1fac7bcd None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:00.484673 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030939c-44fe-4f6f-9c99-cb9b1fac7bcd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:00.485303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030939c-44fe-4f6f-9c99-cb9b1fac7bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:00.507171 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d030939c-44fe-4f6f-9c99-cb9b1fac7bcd None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:00.510066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d030939c-44fe-4f6f-9c99-cb9b1fac7bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:00.518963 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b84272-71ee-47bd-b3ee-793bf627ff5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:00.519667 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b84272-71ee-47bd-b3ee-793bf627ff5b None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:00.520001 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b84272-71ee-47bd-b3ee-793bf627ff5b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:00.520287 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b84272-71ee-47bd-b3ee-793bf627ff5b None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:00.521913 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b84272-71ee-47bd-b3ee-793bf627ff5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:00.523449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b84272-71ee-47bd-b3ee-793bf627ff5b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:00.599516 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 235/467] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:00.601799 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b936320-c59d-4cbe-a49b-dff9c0ebd73d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:00.602326 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b936320-c59d-4cbe-a49b-dff9c0ebd73d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:00.602874 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b936320-c59d-4cbe-a49b-dff9c0ebd73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:00.609484 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 233/468] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jun 7 08:20:00 2022] POST /identity/v3/users => generated 248 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:00.636094 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0b936320-c59d-4cbe-a49b-dff9c0ebd73d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:00.638011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0b936320-c59d-4cbe-a49b-dff9c0ebd73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:00.659332 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b84272-71ee-47bd-b3ee-793bf627ff5b None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:00.717133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:00.746955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 236/469] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:00.782805 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b936320-c59d-4cbe-a49b-dff9c0ebd73d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:00.799871 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b84272-71ee-47bd-b3ee-793bf627ff5b None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:00.843272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:00.845087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a560d5-1ab4-4fd6-9151-8a81cd20ef0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:00.846012 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a560d5-1ab4-4fd6-9151-8a81cd20ef0d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:00.846367 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a560d5-1ab4-4fd6-9151-8a81cd20ef0d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:00.846748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a560d5-1ab4-4fd6-9151-8a81cd20ef0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:00.860227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a560d5-1ab4-4fd6-9151-8a81cd20ef0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:00.861744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a560d5-1ab4-4fd6-9151-8a81cd20ef0d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:00.881941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-54a560d5-1ab4-4fd6-9151-8a81cd20ef0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:00.881941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:00.881941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:00.881941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:00.881941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:00.881941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:00.881941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:00.881941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:00.881941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:00.881941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:00.881941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:00.881941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:00.881941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:00.881941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:00.881941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:00.882835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:00.882835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:00.882835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:00.882835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:00.882835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:00.882835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:00.882835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:00.882835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:00.882835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:00.882835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:00.882835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:00.882835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:00.882835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:00.882835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:00.882835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:00.882835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:00.882835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:00.883683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:00.883683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:00.883683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:00.883683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:00.888062 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 234/470] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:00.889465 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a560d5-1ab4-4fd6-9151-8a81cd20ef0d None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:00.908810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b936320-c59d-4cbe-a49b-dff9c0ebd73d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:00.953468 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fcb8d1-1154-4730-960d-82a5943d8287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:00.954908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcb8d1-1154-4730-960d-82a5943d8287 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:00.955241 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcb8d1-1154-4730-960d-82a5943d8287 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:00.955526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcb8d1-1154-4730-960d-82a5943d8287 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:00.970410 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fcb8d1-1154-4730-960d-82a5943d8287 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:00.972084 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fcb8d1-1154-4730-960d-82a5943d8287 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:00.990854 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:00.997310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a4fcb8d1-1154-4730-960d-82a5943d8287 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:00.997310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:00.997310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:00.997310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:00.997310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:00.997310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:00.997310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:00.997310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:00.997310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:00.997310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:00.997310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:00.997310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:00.997310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:00.997310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:00.997310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:00.998723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:00.998723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:00.998723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:00.998723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:00.998723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:00.998723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:00.998723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:00.998723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:00.998723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:00.998723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:00.998723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:00.998723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:00.998723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:00.998723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:00.998723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:00.998723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:00.998723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:00.999558 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:00.999558 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:00.999558 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:00.999558 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:00.999857 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 237/471] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:01.005255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fcb8d1-1154-4730-960d-82a5943d8287 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:01.016175 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a560d5-1ab4-4fd6-9151-8a81cd20ef0d None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:01.065781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf3c3d-2720-4626-96e4-8a5d90b69c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:01.066753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf3c3d-2720-4626-96e4-8a5d90b69c09 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:01.067219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf3c3d-2720-4626-96e4-8a5d90b69c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:01.067674 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf3c3d-2720-4626-96e4-8a5d90b69c09 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:01.068584 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf3c3d-2720-4626-96e4-8a5d90b69c09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:01.069132 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf3c3d-2720-4626-96e4-8a5d90b69c09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:01.070694 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf3c3d-2720-4626-96e4-8a5d90b69c09 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:01.083902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 235/472] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:01.090264 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf3c3d-2720-4626-96e4-8a5d90b69c09 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:01.127294 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:01.137232 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fcb8d1-1154-4730-960d-82a5943d8287 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:01.175640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf4c33-9f9f-4934-9545-26acbcb9bae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:01.177829 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf4c33-9f9f-4934-9545-26acbcb9bae0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:01.178167 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf4c33-9f9f-4934-9545-26acbcb9bae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:01.178387 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf4c33-9f9f-4934-9545-26acbcb9bae0 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:01.182429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf4c33-9f9f-4934-9545-26acbcb9bae0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:01.182429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf4c33-9f9f-4934-9545-26acbcb9bae0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:01.182978 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf4c33-9f9f-4934-9545-26acbcb9bae0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:01.200323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 238/473] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:01.206160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc447e7e-1967-44d5-a48f-3625e44f5aaa None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:01.206450 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc447e7e-1967-44d5-a48f-3625e44f5aaa None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:01.206732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc447e7e-1967-44d5-a48f-3625e44f5aaa None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:01.208026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 239/474] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:01.208702 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc447e7e-1967-44d5-a48f-3625e44f5aaa None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:01.245303 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf3c3d-2720-4626-96e4-8a5d90b69c09 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:01.282997 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d64fb-2a93-4b11-9988-2a705736f233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:01.284167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d64fb-2a93-4b11-9988-2a705736f233 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:01.284659 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d64fb-2a93-4b11-9988-2a705736f233 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:01.285320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d64fb-2a93-4b11-9988-2a705736f233 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:01.286340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d64fb-2a93-4b11-9988-2a705736f233 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:01.288129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d64fb-2a93-4b11-9988-2a705736f233 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:01.300331 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:01.338900 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc447e7e-1967-44d5-a48f-3625e44f5aaa None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:01.375715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431efac-5b54-43aa-8aa0-f096ffb77370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:01.376722 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431efac-5b54-43aa-8aa0-f096ffb77370 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:01.377262 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431efac-5b54-43aa-8aa0-f096ffb77370 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:01.377753 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431efac-5b54-43aa-8aa0-f096ffb77370 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:01.378748 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431efac-5b54-43aa-8aa0-f096ffb77370 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:01.380553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431efac-5b54-43aa-8aa0-f096ffb77370 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:01.431270 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 236/475] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jun 7 08:20:01 2022] POST /identity/v3/users => generated 249 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:20:01.438943 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc28c1-1bc2-4cc2-ba8d-649fc8e1b5c6 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:01.439244 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc28c1-1bc2-4cc2-ba8d-649fc8e1b5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:01.439519 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc28c1-1bc2-4cc2-ba8d-649fc8e1b5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:01.468160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d8fc28c1-1bc2-4cc2-ba8d-649fc8e1b5c6 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:01.470199 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d8fc28c1-1bc2-4cc2-ba8d-649fc8e1b5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:01.512101 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 240/476] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jun 7 08:20:01 2022] POST /identity/v3/users => generated 248 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:01.570927 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 237/477] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:01.608121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431efac-5b54-43aa-8aa0-f096ffb77370 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:01.619066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:01.741206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431efac-5b54-43aa-8aa0-f096ffb77370 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:01.781108 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a72b90-2368-4102-8542-463eac5c0deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:01.782174 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a72b90-2368-4102-8542-463eac5c0deb None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:01.782660 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a72b90-2368-4102-8542-463eac5c0deb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:01.783134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a72b90-2368-4102-8542-463eac5c0deb None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:01.784111 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a72b90-2368-4102-8542-463eac5c0deb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:01.785754 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a72b90-2368-4102-8542-463eac5c0deb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:01.836390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.sql.core [None req-98a72b90-2368-4102-8542-463eac5c0deb None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 08:20:01.836390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 07 08:20:01.836390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [parameters: {'id': '4a306d2b84454f908bcc91b8ef6b74d8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 07 08:20:01.836390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80358) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 08:20:01.854270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-98a72b90-2368-4102-8542-463eac5c0deb None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 07 08:20:01.854270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:01.854270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 08:20:01.854270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 08:20:01.854270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 08:20:01.854270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 08:20:01.854270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 08:20:01.854270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 08:20:01.854270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 08:20:01.854270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application conn.query(q) Jun 07 08:20:01.854270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 08:20:01.854270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 08:20:01.854270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 08:20:01.854270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result.read() Jun 07 08:20:01.854270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 08:20:01.855221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 08:20:01.855221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 08:20:01.855221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 08:20:01.855221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 08:20:01.855221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 08:20:01.855221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 08:20:01.855221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 08:20:01.855221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 08:20:01.855221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:01.855221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 08:20:01.855221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:01.855221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:01.855221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 08:20:01.855221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 08:20:01.855221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 07 08:20:01.855221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return ref.to_dict() Jun 07 08:20:01.855221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 08:20:01.855221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application next(self.gen) Jun 07 08:20:01.856146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 08:20:01.856146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application yield resource Jun 07 08:20:01.856146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 08:20:01.856146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application next(self.gen) Jun 07 08:20:01.856146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 08:20:01.856146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 08:20:01.856146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 08:20:01.856146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 08:20:01.856146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 08:20:01.856146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise self.value Jun 07 08:20:01.856146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 08:20:01.856146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 08:20:01.856146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 08:20:01.856146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application session.commit() Jun 07 08:20:01.856146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 08:20:01.856146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 08:20:01.856146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 08:20:01.856982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 08:20:01.856982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 08:20:01.856982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.session.flush() Jun 07 08:20:01.856982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 08:20:01.856982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 08:20:01.856982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 08:20:01.856982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 08:20:01.856982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 08:20:01.856982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application compat.raise_( Jun 07 08:20:01.856982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 08:20:01.856982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception Jun 07 08:20:01.856982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 08:20:01.856982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 08:20:01.856982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 08:20:01.856982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 08:20:01.856982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 08:20:01.856982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 08:20:01.857968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 08:20:01.857968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 08:20:01.857968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 07 08:20:01.857968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 07 08:20:01.857968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 08:20:01.857968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 08:20:01.857968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 08:20:01.857968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 08:20:01.857968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 08:20:01.857968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 08:20:01.857968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 08:20:01.857968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 08:20:01.857968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 08:20:01.857968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 08:20:01.857968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 08:20:01.857968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception Jun 07 08:20:01.858765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 08:20:01.858765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 08:20:01.858765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 08:20:01.858765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 08:20:01.858765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 08:20:01.858765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 08:20:01.858765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 08:20:01.858765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application conn.query(q) Jun 07 08:20:01.858765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 08:20:01.858765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 08:20:01.858765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 08:20:01.858765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result.read() Jun 07 08:20:01.858765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 08:20:01.858765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 08:20:01.858765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 08:20:01.858765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 08:20:01.859548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 08:20:01.859548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 08:20:01.859548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 08:20:01.859548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 08:20:01.859548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 08:20:01.859548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 07 08:20:01.859548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [parameters: {'id': '4a306d2b84454f908bcc91b8ef6b74d8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 07 08:20:01.859548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 08:20:01.859548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:01.859548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:01.859548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:01.859548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:01.859548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:01.859548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:01.859548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:01.859548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:01.860350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:01.860350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:01.860350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:01.860350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:01.860350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:01.860350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:01.860350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 07 08:20:01.860350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 07 08:20:01.860350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:01.860350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:01.860350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 07 08:20:01.860350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 07 08:20:01.860350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 08:20:01.860350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 08:20:01.860350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 07 08:20:01.860350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:01.865065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 241/478] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Jun 7 08:20:01 2022] POST /identity/v3/roles => generated 144 bytes in 258 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:01.870694 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc28c1-1bc2-4cc2-ba8d-649fc8e1b5c6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:01.986408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc28c1-1bc2-4cc2-ba8d-649fc8e1b5c6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:02.014195 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:02.023359 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53da3dd-452c-4406-b5b0-64368762f395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:02.026993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53da3dd-452c-4406-b5b0-64368762f395 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:02.026993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53da3dd-452c-4406-b5b0-64368762f395 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:02.026993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53da3dd-452c-4406-b5b0-64368762f395 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:02.037433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53da3dd-452c-4406-b5b0-64368762f395 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:02.045533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-d53da3dd-452c-4406-b5b0-64368762f395 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:20:02.045533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:02.045533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:02.045533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:02.045533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:02.045533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:02.045533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:02.045533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:02.045533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:02.045533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:02.045533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:02.045533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:02.045533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:02.045533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:02.045533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:02.045533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:02.046505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:02.046505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:02.046505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:02.046505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:02.046505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:02.046505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:02.046505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:02.046505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:02.046505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:02.046505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:02.046505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:02.046505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:02.046505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:02.046505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:02.046505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:02.046505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:02.047296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:02.047296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:02.047296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:02.047296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:02.047296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:02.047296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:02.047296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:02.047296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:02.047296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:02.047296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:02.047296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 08:20:02.047296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:02.047296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 238/479] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:20:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:02.062083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a72b90-2368-4102-8542-463eac5c0deb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:02.164743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:02.207816 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a72b90-2368-4102-8542-463eac5c0deb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:02.243215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d147b-961e-4400-86ac-af7c619a0879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:02.244249 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d147b-961e-4400-86ac-af7c619a0879 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:02.244732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d147b-961e-4400-86ac-af7c619a0879 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:02.245323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d147b-961e-4400-86ac-af7c619a0879 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:02.246271 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d147b-961e-4400-86ac-af7c619a0879 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:02.246815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d147b-961e-4400-86ac-af7c619a0879 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:02.248930 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d147b-961e-4400-86ac-af7c619a0879 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:02.257862 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 242/480] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:03.127750 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348e4d3-c634-4a3b-af4b-5c677e9ff3da None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:03.128262 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348e4d3-c634-4a3b-af4b-5c677e9ff3da None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:03.128262 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348e4d3-c634-4a3b-af4b-5c677e9ff3da None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:03.129907 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 239/481] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:03.135344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4a11e-5051-48d4-8831-90997811b3a8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:03.135731 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4a11e-5051-48d4-8831-90997811b3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:03.135894 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4a11e-5051-48d4-8831-90997811b3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:03.161996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e4e4a11e-5051-48d4-8831-90997811b3a8 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:03.163188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e4e4a11e-5051-48d4-8831-90997811b3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:03.260430 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 243/482] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:20:03.297452 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6348e4d3-c634-4a3b-af4b-5c677e9ff3da None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:03.422671 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6348e4d3-c634-4a3b-af4b-5c677e9ff3da None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:03.468927 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15609e4e-6549-4633-aab9-4e9eb1794536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:03.469741 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15609e4e-6549-4633-aab9-4e9eb1794536 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:03.470017 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15609e4e-6549-4633-aab9-4e9eb1794536 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:03.470346 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15609e4e-6549-4633-aab9-4e9eb1794536 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:03.484286 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15609e4e-6549-4633-aab9-4e9eb1794536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:03.492122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-15609e4e-6549-4633-aab9-4e9eb1794536 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:03.492122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:03.492122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:03.492122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:03.492122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:03.492122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:03.492122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:03.492122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:03.492122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:03.492122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:03.492122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:03.492122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:03.492122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:03.492122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:03.492122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:03.492122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:03.492980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:03.492980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:03.492980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:03.492980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:03.492980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:03.492980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:03.492980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:03.492980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:03.492980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:03.492980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:03.492980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:03.492980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:03.492980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:03.492980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:03.492980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:03.492980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:03.494233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:03.494233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:03.494233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:03.494233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:03.494233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:03.494233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:03.494233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:03.494233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:03.494233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:03.494233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:03.494233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:03.494233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:03.494233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 240/483] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:03.499988 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4a11e-5051-48d4-8831-90997811b3a8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:03.578860 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:03.618321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4a11e-5051-48d4-8831-90997811b3a8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:03.653472 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930ebe7-3744-4c53-ba97-e4f36fde63ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:03.654133 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930ebe7-3744-4c53-ba97-e4f36fde63ab None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:03.654444 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930ebe7-3744-4c53-ba97-e4f36fde63ab None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:03.654716 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930ebe7-3744-4c53-ba97-e4f36fde63ab None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:03.655461 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930ebe7-3744-4c53-ba97-e4f36fde63ab None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:03.655762 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930ebe7-3744-4c53-ba97-e4f36fde63ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:03.657152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930ebe7-3744-4c53-ba97-e4f36fde63ab None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:03.665811 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 244/484] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:20:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:03.671240 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15609e4e-6549-4633-aab9-4e9eb1794536 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:03.722912 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:03.731970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe4e12-8d7e-4564-8cf9-537351e1941e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:03.732256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe4e12-8d7e-4564-8cf9-537351e1941e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:03.732697 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe4e12-8d7e-4564-8cf9-537351e1941e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:03.734251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 245/485] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:20:03.739455 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccda433-edc1-4206-8938-835f7133fcd1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:03.739741 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccda433-edc1-4206-8938-835f7133fcd1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:03.740021 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccda433-edc1-4206-8938-835f7133fcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:03.774045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1ccda433-edc1-4206-8938-835f7133fcd1 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:03.775256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1ccda433-edc1-4206-8938-835f7133fcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:03.787783 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15609e4e-6549-4633-aab9-4e9eb1794536 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:03.830618 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f0014-bed7-4d16-b10f-987777b2295a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:03.831479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f0014-bed7-4d16-b10f-987777b2295a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:03.831755 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f0014-bed7-4d16-b10f-987777b2295a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:03.832027 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f0014-bed7-4d16-b10f-987777b2295a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:03.844687 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f0014-bed7-4d16-b10f-987777b2295a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:03.846332 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f0014-bed7-4d16-b10f-987777b2295a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:03.866297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-e33f0014-bed7-4d16-b10f-987777b2295a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:03.866297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:03.866297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:03.866297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:03.866297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:03.866297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:03.866297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:03.866297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:03.866297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:03.866297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:03.866297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:03.866297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:03.866297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:03.866297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:03.866297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:03.868389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:03.868389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:03.868389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:03.868389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:03.868389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:03.868389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:03.868389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:03.868389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:03.868389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:03.868389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:03.868389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:03.868389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:03.868389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:03.868389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:03.868389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:03.868389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:03.868389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:03.870538 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:03.870538 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:03.870538 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:03.870538 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:03.870538 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 241/486] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:03.873596 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f0014-bed7-4d16-b10f-987777b2295a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:03.979384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:03.984523 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f0014-bed7-4d16-b10f-987777b2295a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:04.015236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 246/487] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:04.020282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20d464-0d65-4326-b6a8-d9fe651bec38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:04.020933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20d464-0d65-4326-b6a8-d9fe651bec38 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:04.021240 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20d464-0d65-4326-b6a8-d9fe651bec38 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:04.021513 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20d464-0d65-4326-b6a8-d9fe651bec38 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:04.022233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20d464-0d65-4326-b6a8-d9fe651bec38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:04.022535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20d464-0d65-4326-b6a8-d9fe651bec38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:04.023964 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20d464-0d65-4326-b6a8-d9fe651bec38 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:04.037064 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 242/488] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:04.042417 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccda433-edc1-4206-8938-835f7133fcd1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:04.059839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20d464-0d65-4326-b6a8-d9fe651bec38 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:04.241574 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20d464-0d65-4326-b6a8-d9fe651bec38 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:04.281852 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be628e-f338-4009-af09-95367db6b3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:04.283341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be628e-f338-4009-af09-95367db6b3bc None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:04.283851 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be628e-f338-4009-af09-95367db6b3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:04.284304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be628e-f338-4009-af09-95367db6b3bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:04.296185 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be628e-f338-4009-af09-95367db6b3bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:04.304665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-66be628e-f338-4009-af09-95367db6b3bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:04.304665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:04.304665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:04.304665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:04.304665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:04.304665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:04.304665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:04.304665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:04.304665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:04.304665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:04.304665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:04.304665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:04.304665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:04.304665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:04.304665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:04.304665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:04.305569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:04.305569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:04.305569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:04.305569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:04.305569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:04.305569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:04.305569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:04.305569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:04.305569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:04.305569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:04.305569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:04.305569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:04.305569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:04.305569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:04.305569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:04.305569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:04.306344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:04.306344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:04.306344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:04.306344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:04.306344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:04.306344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:04.306344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:04.306344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:04.306344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:04.306344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:04.306344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:04.306344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:04.309074 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 243/489] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:04.309396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:04.311262 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4845ad9-f3d6-474a-9618-3dae5b326315 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:04.311778 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4845ad9-f3d6-474a-9618-3dae5b326315 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:04.312247 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4845ad9-f3d6-474a-9618-3dae5b326315 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:04.314098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 244/490] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:04.315037 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4845ad9-f3d6-474a-9618-3dae5b326315 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:04.335829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:04.345444 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccda433-edc1-4206-8938-835f7133fcd1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:04.381186 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298224c2-99ea-48aa-af2e-e327d04a3725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:04.382356 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298224c2-99ea-48aa-af2e-e327d04a3725 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:04.382599 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298224c2-99ea-48aa-af2e-e327d04a3725 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:04.382880 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298224c2-99ea-48aa-af2e-e327d04a3725 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:04.390403 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298224c2-99ea-48aa-af2e-e327d04a3725 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:04.391853 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298224c2-99ea-48aa-af2e-e327d04a3725 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:04.408151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-298224c2-99ea-48aa-af2e-e327d04a3725 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:20:04.408151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:04.408151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:04.408151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:04.408151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:04.408151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:04.408151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:04.408151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:04.408151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:04.408151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:04.408151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:04.408151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:04.408151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:04.408151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:04.408151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:04.408151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:04.409098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:04.409098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:04.409098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:04.409098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:04.409098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:04.409098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:04.409098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:04.409098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:04.409098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:04.409098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:04.409098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:04.409098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:04.409098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:04.409098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:04.409098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:04.409098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:04.409098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:04.409098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:04.409974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:04.409974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:04.409974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:04.409974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:04.409974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:04.409974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:04.409974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:04.409974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:04.409974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:20:04.409974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:04.413987 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 247/491] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Jun 7 08:20:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:04.415265 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db10969-4ec3-431f-a339-33ae145ac90f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:04.415558 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db10969-4ec3-431f-a339-33ae145ac90f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:04.415841 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db10969-4ec3-431f-a339-33ae145ac90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:04.430891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4845ad9-f3d6-474a-9618-3dae5b326315 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:04.434167 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3db10969-4ec3-431f-a339-33ae145ac90f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:04.434824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3db10969-4ec3-431f-a339-33ae145ac90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:04.466311 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db5f96-2f5e-4175-8e33-6dac5096a0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:04.467518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db5f96-2f5e-4175-8e33-6dac5096a0fd None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:04.467805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db5f96-2f5e-4175-8e33-6dac5096a0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:04.468082 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db5f96-2f5e-4175-8e33-6dac5096a0fd None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:04.468829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db5f96-2f5e-4175-8e33-6dac5096a0fd None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:04.469165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db5f96-2f5e-4175-8e33-6dac5096a0fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:04.470541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db5f96-2f5e-4175-8e33-6dac5096a0fd None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:04.478754 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 245/492] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:20:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:04.479385 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db5f96-2f5e-4175-8e33-6dac5096a0fd None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:04.509114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 248/493] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:04.514689 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ae3c8-84ad-4a1a-b4d3-156d9c3d0a45 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:04.515141 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ae3c8-84ad-4a1a-b4d3-156d9c3d0a45 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:04.515789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ae3c8-84ad-4a1a-b4d3-156d9c3d0a45 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:04.518098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 249/494] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:04.518928 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ae3c8-84ad-4a1a-b4d3-156d9c3d0a45 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:04.565498 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:04.573749 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:04.586120 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db5f96-2f5e-4175-8e33-6dac5096a0fd None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:04.623000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f24489-d459-497b-a77c-6abf4236c976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:04.623797 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f24489-d459-497b-a77c-6abf4236c976 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:04.624173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f24489-d459-497b-a77c-6abf4236c976 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:04.624517 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f24489-d459-497b-a77c-6abf4236c976 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:04.625380 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f24489-d459-497b-a77c-6abf4236c976 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:04.625690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f24489-d459-497b-a77c-6abf4236c976 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:04.627274 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f24489-d459-497b-a77c-6abf4236c976 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:04.630920 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ae3c8-84ad-4a1a-b4d3-156d9c3d0a45 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:04.664749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 246/495] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Jun 7 08:20:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:04.666088 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779b380-2a7f-4bef-9db6-811fc8c3bb1b None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:04.666372 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779b380-2a7f-4bef-9db6-811fc8c3bb1b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:04.666645 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779b380-2a7f-4bef-9db6-811fc8c3bb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:04.670630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a962e-b7c6-41ba-a89f-53a7063c02ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:04.671669 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a962e-b7c6-41ba-a89f-53a7063c02ca None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:04.672203 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a962e-b7c6-41ba-a89f-53a7063c02ca None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:04.672929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a962e-b7c6-41ba-a89f-53a7063c02ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:04.684333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-2779b380-2a7f-4bef-9db6-811fc8c3bb1b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:04.684927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a962e-b7c6-41ba-a89f-53a7063c02ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:04.685563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2779b380-2a7f-4bef-9db6-811fc8c3bb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:04.686493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a962e-b7c6-41ba-a89f-53a7063c02ca None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:04.704776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-6c3a962e-b7c6-41ba-a89f-53a7063c02ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:04.704776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:04.704776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:04.704776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:04.704776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:04.704776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:04.704776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:04.704776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:04.704776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:04.704776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:04.704776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:04.704776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:04.704776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:04.704776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:04.704776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:04.705649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:04.705649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:04.705649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:04.705649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:04.705649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:04.705649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:04.705649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:04.705649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:04.705649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:04.705649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:04.705649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:04.705649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:04.705649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:04.705649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:04.705649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:04.705649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:04.705649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:04.706493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:04.706493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:04.706493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:04.706493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:04.706493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 250/496] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:04.706761 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a962e-b7c6-41ba-a89f-53a7063c02ca None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:04.769234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 247/497] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:04.769234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779b380-2a7f-4bef-9db6-811fc8c3bb1b None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:04.806987 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:04.812662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a962e-b7c6-41ba-a89f-53a7063c02ca None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:04.848829 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8781b9-706b-4335-a1c0-007a05c79637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:04.849956 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:04.850612 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8781b9-706b-4335-a1c0-007a05c79637 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:04.851034 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8781b9-706b-4335-a1c0-007a05c79637 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:04.851571 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8781b9-706b-4335-a1c0-007a05c79637 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:04.862952 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8781b9-706b-4335-a1c0-007a05c79637 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:04.870752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1f8781b9-706b-4335-a1c0-007a05c79637 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:04.870752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:04.870752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:04.870752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:04.870752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:04.870752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:04.870752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:04.870752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:04.870752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:04.870752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:04.870752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:04.870752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:04.870752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:04.870752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:04.870752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:04.870752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:04.871781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:04.871781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:04.871781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:04.871781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:04.871781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:04.871781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:04.871781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:04.871781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:04.871781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:04.871781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:04.871781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:04.871781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:04.871781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:04.871781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:04.871781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:04.871781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:04.872746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:04.872746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:04.872746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:04.872746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:04.872746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:04.872746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:04.872746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:04.872746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:04.872746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:04.872746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:04.872746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:04.872746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:04.874295 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 251/498] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:04.875081 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8781b9-706b-4335-a1c0-007a05c79637 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:04.887445 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779b380-2a7f-4bef-9db6-811fc8c3bb1b None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:04.921237 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3a071-0236-484e-85bc-295fc1dce37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:04.921885 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3a071-0236-484e-85bc-295fc1dce37e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:04.922171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3a071-0236-484e-85bc-295fc1dce37e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:04.922446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3a071-0236-484e-85bc-295fc1dce37e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:04.983867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8781b9-706b-4335-a1c0-007a05c79637 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:04.991685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3a071-0236-484e-85bc-295fc1dce37e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:04.993178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3a071-0236-484e-85bc-295fc1dce37e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:05.020383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4eddd-831d-4f30-9a2a-37e3fb0ea10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:05.021112 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4eddd-831d-4f30-9a2a-37e3fb0ea10b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:05.021512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4eddd-831d-4f30-9a2a-37e3fb0ea10b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:05.021793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4eddd-831d-4f30-9a2a-37e3fb0ea10b None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:05.022547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4eddd-831d-4f30-9a2a-37e3fb0ea10b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:05.022860 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4eddd-831d-4f30-9a2a-37e3fb0ea10b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:05.024420 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4eddd-831d-4f30-9a2a-37e3fb0ea10b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:05.043565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 252/499] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:05.043565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4eddd-831d-4f30-9a2a-37e3fb0ea10b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:05.060371 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:05.069190 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:05.103479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 248/500] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:20:05.104396 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3a071-0236-484e-85bc-295fc1dce37e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:05.147863 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4eddd-831d-4f30-9a2a-37e3fb0ea10b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:05.187179 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e8589-ced4-45ef-b74c-b5c2a7264a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:05.188256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e8589-ced4-45ef-b74c-b5c2a7264a1d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:05.188256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e8589-ced4-45ef-b74c-b5c2a7264a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:05.188509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e8589-ced4-45ef-b74c-b5c2a7264a1d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:05.199080 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e8589-ced4-45ef-b74c-b5c2a7264a1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:05.206604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-206e8589-ced4-45ef-b74c-b5c2a7264a1d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:05.206604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:05.206604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:05.206604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:05.206604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:05.206604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:05.206604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:05.206604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:05.206604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:05.206604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:05.206604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:05.206604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:05.206604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:05.206604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:05.206604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:05.206604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:05.207465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:05.207465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:05.207465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:05.207465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:05.207465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:05.207465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:05.207465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:05.207465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:05.207465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:05.207465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:05.207465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:05.207465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:05.207465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:05.207465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:05.207465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:05.207465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:05.208286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:05.208286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:05.208286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:05.208286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:05.208286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:05.208286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:05.208286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:05.208286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:05.208286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:05.208286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:05.208286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:05.208286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:05.208286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 253/501] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:05.208945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e8589-ced4-45ef-b74c-b5c2a7264a1d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:05.222911 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3a071-0236-484e-85bc-295fc1dce37e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:05.260847 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84839cb0-1d7c-4005-8ba0-6ce5acd2578c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:05.261793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84839cb0-1d7c-4005-8ba0-6ce5acd2578c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:05.262090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84839cb0-1d7c-4005-8ba0-6ce5acd2578c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:05.262372 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84839cb0-1d7c-4005-8ba0-6ce5acd2578c None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:05.263086 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84839cb0-1d7c-4005-8ba0-6ce5acd2578c None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:05.263387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84839cb0-1d7c-4005-8ba0-6ce5acd2578c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:05.264796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84839cb0-1d7c-4005-8ba0-6ce5acd2578c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:05.273074 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 249/502] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:20:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:05.278174 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84839cb0-1d7c-4005-8ba0-6ce5acd2578c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:05.290233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:05.310334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:05.314305 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e8589-ced4-45ef-b74c-b5c2a7264a1d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:05.352004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef3d31-4ca4-4850-bb9e-9be3ecdda431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:05.352809 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef3d31-4ca4-4850-bb9e-9be3ecdda431 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:05.353123 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef3d31-4ca4-4850-bb9e-9be3ecdda431 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:05.353405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef3d31-4ca4-4850-bb9e-9be3ecdda431 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:05.362953 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef3d31-4ca4-4850-bb9e-9be3ecdda431 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:05.364396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef3d31-4ca4-4850-bb9e-9be3ecdda431 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:05.378543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-72ef3d31-4ca4-4850-bb9e-9be3ecdda431 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 08:20:05.378543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:05.378543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:05.378543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:05.378543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:05.378543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:05.378543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:05.378543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:05.378543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:05.378543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:05.378543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:05.378543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:05.378543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:05.378543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:05.378543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:05.378543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:05.379561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:05.379561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:05.379561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:05.379561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:05.379561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:05.379561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:05.379561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:05.379561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:05.379561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:05.379561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:05.379561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:05.379561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:05.379561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:05.379561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:05.379561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:05.379561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:05.379561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:05.379561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:05.380458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:05.380458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:05.380458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:05.380458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:05.380458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:05.380458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:05.380458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:05.380458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:05.380458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 08:20:05.380458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:05.380458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 254/503] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:05.381311 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84839cb0-1d7c-4005-8ba0-6ce5acd2578c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:05.381416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef3d31-4ca4-4850-bb9e-9be3ecdda431 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:05.416106 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37656965-d2ff-4e34-b52b-3e099d80cfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:05.416887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37656965-d2ff-4e34-b52b-3e099d80cfab None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:05.417204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37656965-d2ff-4e34-b52b-3e099d80cfab None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:05.417478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37656965-d2ff-4e34-b52b-3e099d80cfab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:05.428101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37656965-d2ff-4e34-b52b-3e099d80cfab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:05.429564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37656965-d2ff-4e34-b52b-3e099d80cfab None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:05.445811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-37656965-d2ff-4e34-b52b-3e099d80cfab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:05.445811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:05.445811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:05.445811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:05.445811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:05.445811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:05.445811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:05.445811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:05.445811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:05.445811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:05.445811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:05.445811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:05.445811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:05.445811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:05.445811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:05.446781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:05.446781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:05.446781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:05.446781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:05.446781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:05.446781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:05.446781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:05.446781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:05.446781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:05.446781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:05.446781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:05.446781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:05.446781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:05.446781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:05.446781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:05.446781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:05.446781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:05.447621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:05.447621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:05.447621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:05.447621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:05.447621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 250/504] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:05.447884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37656965-d2ff-4e34-b52b-3e099d80cfab None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:05.480215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef3d31-4ca4-4850-bb9e-9be3ecdda431 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:05.512928 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a5334-1cc7-417c-8c98-a819b7363923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:05.514222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a5334-1cc7-417c-8c98-a819b7363923 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:05.514569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a5334-1cc7-417c-8c98-a819b7363923 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:05.514848 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a5334-1cc7-417c-8c98-a819b7363923 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:05.515583 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a5334-1cc7-417c-8c98-a819b7363923 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:05.515884 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a5334-1cc7-417c-8c98-a819b7363923 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:05.517252 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a5334-1cc7-417c-8c98-a819b7363923 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:05.519334 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:05.533095 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 255/505] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:20:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:05.533251 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:05.533346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616bfe8-c853-46ef-8912-204c246b7153 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:05.533346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616bfe8-c853-46ef-8912-204c246b7153 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:05.533346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616bfe8-c853-46ef-8912-204c246b7153 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:05.533346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 256/506] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:05.533346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616bfe8-c853-46ef-8912-204c246b7153 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:05.548872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37656965-d2ff-4e34-b52b-3e099d80cfab None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:05.581984 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed48c1-b0de-4546-a67d-94647c159f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:05.582762 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed48c1-b0de-4546-a67d-94647c159f55 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:05.583041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed48c1-b0de-4546-a67d-94647c159f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:05.583313 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed48c1-b0de-4546-a67d-94647c159f55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:05.594095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed48c1-b0de-4546-a67d-94647c159f55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:05.595531 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed48c1-b0de-4546-a67d-94647c159f55 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:05.612094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-97ed48c1-b0de-4546-a67d-94647c159f55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:05.612094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:05.612094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:05.612094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:05.612094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:05.612094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:05.612094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:05.612094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:05.612094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:05.612094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:05.612094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:05.612094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:05.612094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:05.612094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:05.612094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:05.613133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:05.613133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:05.613133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:05.613133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:05.613133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:05.613133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:05.613133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:05.613133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:05.613133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:05.613133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:05.613133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:05.613133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:05.613133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:05.613133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:05.613133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:05.613133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:05.613133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:05.614005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:05.614005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:05.614005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:05.614005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:05.614005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 251/507] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:05.614256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed48c1-b0de-4546-a67d-94647c159f55 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:05.630278 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616bfe8-c853-46ef-8912-204c246b7153 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:05.663780 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a2c96-869e-472b-9890-a2448b979e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:05.664448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a2c96-869e-472b-9890-a2448b979e52 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:05.664732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a2c96-869e-472b-9890-a2448b979e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:05.665029 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a2c96-869e-472b-9890-a2448b979e52 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:05.665861 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a2c96-869e-472b-9890-a2448b979e52 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:05.666171 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a2c96-869e-472b-9890-a2448b979e52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:05.667592 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a2c96-869e-472b-9890-a2448b979e52 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:05.703412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 257/508] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Jun 7 08:20:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:05.704225 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a2c96-869e-472b-9890-a2448b979e52 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:05.714205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed48c1-b0de-4546-a67d-94647c159f55 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:05.746803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3486da-cf6d-4e18-9bb7-79e4132c808f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:05.747447 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3486da-cf6d-4e18-9bb7-79e4132c808f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:05.747723 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3486da-cf6d-4e18-9bb7-79e4132c808f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:05.748531 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3486da-cf6d-4e18-9bb7-79e4132c808f None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:05.749272 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3486da-cf6d-4e18-9bb7-79e4132c808f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:05.749577 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3486da-cf6d-4e18-9bb7-79e4132c808f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:05.750938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3486da-cf6d-4e18-9bb7-79e4132c808f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:05.752307 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:05.758448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:05.762956 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 252/509] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:05.764733 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904aaa1-4462-44c8-9967-450e4741cd52 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:05.765033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904aaa1-4462-44c8-9967-450e4741cd52 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:05.765311 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904aaa1-4462-44c8-9967-450e4741cd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:05.780668 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-f904aaa1-4462-44c8-9967-450e4741cd52 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:05.781897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f904aaa1-4462-44c8-9967-450e4741cd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:05.803099 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a2c96-869e-472b-9890-a2448b979e52 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:05.836184 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb65d0-4620-4cb5-9534-ed592f08fd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:05.837067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb65d0-4620-4cb5-9534-ed592f08fd34 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:05.837348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb65d0-4620-4cb5-9534-ed592f08fd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:05.837619 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb65d0-4620-4cb5-9534-ed592f08fd34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:05.846267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 253/510] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:05.847117 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904aaa1-4462-44c8-9967-450e4741cd52 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:05.847726 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb65d0-4620-4cb5-9534-ed592f08fd34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:05.849181 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb65d0-4620-4cb5-9534-ed592f08fd34 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:05.865958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d4bb65d0-4620-4cb5-9534-ed592f08fd34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:05.865958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:05.865958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:05.865958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:05.865958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:05.865958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:05.865958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:05.865958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:05.865958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:05.865958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:05.865958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:05.865958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:05.865958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:05.865958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:05.865958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:05.866818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:05.866818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:05.866818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:05.866818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:05.866818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:05.866818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:05.866818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:05.866818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:05.866818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:05.866818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:05.866818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:05.866818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:05.866818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:05.866818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:05.866818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:05.866818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:05.866818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:05.867996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:05.867996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:05.867996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:05.867996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:05.867996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 258/511] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:05.868451 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb65d0-4620-4cb5-9534-ed592f08fd34 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:05.954399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904aaa1-4462-44c8-9967-450e4741cd52 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:05.965301 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb65d0-4620-4cb5-9534-ed592f08fd34 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:05.974497 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:05.976320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:05.986623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf73569-9fca-4d39-9ed2-8a9db3a372e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:05.987285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf73569-9fca-4d39-9ed2-8a9db3a372e5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:05.987569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf73569-9fca-4d39-9ed2-8a9db3a372e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:05.987890 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf73569-9fca-4d39-9ed2-8a9db3a372e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:05.988557 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf73569-9fca-4d39-9ed2-8a9db3a372e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:05.988859 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf73569-9fca-4d39-9ed2-8a9db3a372e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:05.990334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf73569-9fca-4d39-9ed2-8a9db3a372e5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:05.998921 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739f117-7fe2-4147-8a3a-82b29f59ee8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:05.999558 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739f117-7fe2-4147-8a3a-82b29f59ee8e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:05.999846 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739f117-7fe2-4147-8a3a-82b29f59ee8e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:06.000123 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739f117-7fe2-4147-8a3a-82b29f59ee8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:06.001713 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 254/512] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:06.002341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf73569-9fca-4d39-9ed2-8a9db3a372e5 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:06.068431 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739f117-7fe2-4147-8a3a-82b29f59ee8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:06.069830 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739f117-7fe2-4147-8a3a-82b29f59ee8e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:06.101532 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf73569-9fca-4d39-9ed2-8a9db3a372e5 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:06.134206 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916deb9-3980-416f-9349-17dfc7adc885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:06.134982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916deb9-3980-416f-9349-17dfc7adc885 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:06.135265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916deb9-3980-416f-9349-17dfc7adc885 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:06.135539 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916deb9-3980-416f-9349-17dfc7adc885 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:06.142050 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916deb9-3980-416f-9349-17dfc7adc885 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:06.143473 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916deb9-3980-416f-9349-17dfc7adc885 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:06.158217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-a916deb9-3980-416f-9349-17dfc7adc885 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:06.158217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:06.158217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:06.158217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:06.158217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:06.158217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:06.158217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:06.158217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:06.158217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:06.158217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:06.158217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:06.158217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:06.158217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:06.158217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:06.158217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:06.159161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:06.159161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:06.159161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:06.159161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:06.159161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:06.159161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:06.159161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:06.159161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:06.159161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:06.159161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:06.159161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:06.159161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:06.159161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:06.159161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:06.159161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:06.159161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:06.159161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:06.160018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:06.160018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:06.160018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:06.160018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:06.160018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:06.160018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:06.160018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:06.160018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:06.160018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 255/513] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:06.160477 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916deb9-3980-416f-9349-17dfc7adc885 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:06.170980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 259/514] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:20:06.172231 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739f117-7fe2-4147-8a3a-82b29f59ee8e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:06.182229 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:06.213173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:06.303886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739f117-7fe2-4147-8a3a-82b29f59ee8e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:06.304084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916deb9-3980-416f-9349-17dfc7adc885 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:06.336658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899805ca-763a-4d60-8883-a52d1422f442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:06.337176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd649dee-b79f-48e4-9d36-877a01deb34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:06.337281 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899805ca-763a-4d60-8883-a52d1422f442 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:06.337479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd649dee-b79f-48e4-9d36-877a01deb34a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:06.337569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899805ca-763a-4d60-8883-a52d1422f442 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:06.337759 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd649dee-b79f-48e4-9d36-877a01deb34a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:06.337849 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899805ca-763a-4d60-8883-a52d1422f442 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:06.338028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd649dee-b79f-48e4-9d36-877a01deb34a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:06.338571 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899805ca-763a-4d60-8883-a52d1422f442 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:06.338820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899805ca-763a-4d60-8883-a52d1422f442 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:06.340188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899805ca-763a-4d60-8883-a52d1422f442 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:06.347839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd649dee-b79f-48e4-9d36-877a01deb34a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:06.349301 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd649dee-b79f-48e4-9d36-877a01deb34a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:06.351241 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 260/515] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:06.352029 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899805ca-763a-4d60-8883-a52d1422f442 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:06.371518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-cd649dee-b79f-48e4-9d36-877a01deb34a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:06.371518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:06.371518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:06.371518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:06.371518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:06.371518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:06.371518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:06.371518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:06.371518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:06.371518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:06.371518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:06.371518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:06.371518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:06.371518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:06.371518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:06.372544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:06.372544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:06.372544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:06.372544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:06.372544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:06.372544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:06.372544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:06.372544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:06.372544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:06.372544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:06.372544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:06.372544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:06.372544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:06.372544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:06.372544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:06.372544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:06.372544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:06.374054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:06.374054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:06.374054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:06.374054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:06.377641 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 256/516] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:06.378588 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd649dee-b79f-48e4-9d36-877a01deb34a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:06.417121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:06.451461 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899805ca-763a-4d60-8883-a52d1422f442 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:06.468710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:06.474311 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd649dee-b79f-48e4-9d36-877a01deb34a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:06.484646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e074a0-5504-4885-ac71-28a1f1dbf5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:06.485474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e074a0-5504-4885-ac71-28a1f1dbf5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:06.485758 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e074a0-5504-4885-ac71-28a1f1dbf5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:06.486035 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e074a0-5504-4885-ac71-28a1f1dbf5c6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:06.493619 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e074a0-5504-4885-ac71-28a1f1dbf5c6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:06.495061 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e074a0-5504-4885-ac71-28a1f1dbf5c6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:06.507795 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee897e-23c7-4110-8f02-e1c00485416f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:06.508438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee897e-23c7-4110-8f02-e1c00485416f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:06.508716 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee897e-23c7-4110-8f02-e1c00485416f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:06.508810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-41e074a0-5504-4885-ac71-28a1f1dbf5c6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 08:20:06.508810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:06.508810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:06.508810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:06.508810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:06.508810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:06.508810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:06.508810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:06.508810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:06.508810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:06.508810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:06.508810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:06.508810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:06.508810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:06.508810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:06.508810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:06.509864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:06.509864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:06.509864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:06.509864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:06.509864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:06.509864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:06.509864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:06.509864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:06.509864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:06.509864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:06.509864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:06.509864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:06.509864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:06.509864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:06.509864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:06.509864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:06.509864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:06.509864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:06.511071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:06.511071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:06.511071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:06.511071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:06.511071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:06.511071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:06.511071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:06.511071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:06.511071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 08:20:06.511071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:06.511594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee897e-23c7-4110-8f02-e1c00485416f None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:06.511594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee897e-23c7-4110-8f02-e1c00485416f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:06.511594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee897e-23c7-4110-8f02-e1c00485416f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:06.511775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 261/517] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:06.511775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e074a0-5504-4885-ac71-28a1f1dbf5c6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:06.511890 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee897e-23c7-4110-8f02-e1c00485416f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:06.528542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 257/518] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:06.530027 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee897e-23c7-4110-8f02-e1c00485416f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:06.612495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e074a0-5504-4885-ac71-28a1f1dbf5c6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:06.628244 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee897e-23c7-4110-8f02-e1c00485416f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:06.637205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:06.644813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d719574a-cc30-4f52-868f-273d56d725bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:06.649162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719574a-cc30-4f52-868f-273d56d725bb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:06.649162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719574a-cc30-4f52-868f-273d56d725bb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:06.649162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719574a-cc30-4f52-868f-273d56d725bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:06.656719 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d719574a-cc30-4f52-868f-273d56d725bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:06.658244 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d719574a-cc30-4f52-868f-273d56d725bb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:06.660968 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4592e7-7b79-40d3-a56a-a83a1e03e3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:06.661772 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4592e7-7b79-40d3-a56a-a83a1e03e3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:06.662048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4592e7-7b79-40d3-a56a-a83a1e03e3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:06.662319 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4592e7-7b79-40d3-a56a-a83a1e03e3e0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:06.669593 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4592e7-7b79-40d3-a56a-a83a1e03e3e0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:06.671583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4592e7-7b79-40d3-a56a-a83a1e03e3e0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:06.676640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d719574a-cc30-4f52-868f-273d56d725bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:06.676640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:06.676640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:06.676640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:06.676640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:06.676640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:06.676640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:06.676640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:06.676640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:06.676640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:06.676640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:06.676640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:06.676640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:06.676640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:06.676640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:06.677648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:06.677648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:06.677648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:06.677648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:06.677648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:06.677648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:06.677648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:06.677648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:06.677648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:06.677648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:06.677648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:06.677648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:06.677648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:06.677648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:06.677648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:06.677648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:06.677648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:06.678693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:06.678693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:06.678693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:06.678693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:06.678693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 262/519] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:06.678693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d719574a-cc30-4f52-868f-273d56d725bb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:06.688638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-6b4592e7-7b79-40d3-a56a-a83a1e03e3e0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 08:20:06.688638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:06.688638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:06.688638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:06.688638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:06.688638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:06.688638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:06.688638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:06.688638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:06.688638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:06.688638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:06.688638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:06.688638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:06.688638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:06.688638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:06.688638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:06.689747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:06.689747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:06.689747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:06.689747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:06.689747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:06.689747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:06.689747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:06.689747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:06.689747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:06.689747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:06.689747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:06.689747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:06.689747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:06.689747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:06.689747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:06.689747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:06.689747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:06.689747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:06.691020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:06.691020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:06.691020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:06.691020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:06.691020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:06.691020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:06.691020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:06.691020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:06.691020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 08:20:06.691020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:06.691020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 258/520] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:06.691020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4592e7-7b79-40d3-a56a-a83a1e03e3e0 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:06.708801 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:06.778938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d719574a-cc30-4f52-868f-273d56d725bb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:06.788538 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4592e7-7b79-40d3-a56a-a83a1e03e3e0 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:06.812521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bc23b-fc76-4543-be8c-70c2e92d20b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:06.813208 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bc23b-fc76-4543-be8c-70c2e92d20b2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:06.813485 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bc23b-fc76-4543-be8c-70c2e92d20b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:06.813762 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bc23b-fc76-4543-be8c-70c2e92d20b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:06.814496 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bc23b-fc76-4543-be8c-70c2e92d20b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:06.814798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bc23b-fc76-4543-be8c-70c2e92d20b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:06.816158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bc23b-fc76-4543-be8c-70c2e92d20b2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:06.821623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410af7b9-32da-4978-9871-8ff047ef2a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:06.822279 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410af7b9-32da-4978-9871-8ff047ef2a22 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:06.822571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410af7b9-32da-4978-9871-8ff047ef2a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:06.822850 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410af7b9-32da-4978-9871-8ff047ef2a22 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:06.823683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410af7b9-32da-4978-9871-8ff047ef2a22 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:06.823993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410af7b9-32da-4978-9871-8ff047ef2a22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:06.825451 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410af7b9-32da-4978-9871-8ff047ef2a22 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:06.827321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 263/521] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:06.828112 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bc23b-fc76-4543-be8c-70c2e92d20b2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:06.847818 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 259/522] 10.222.0.17 () {56 vars in 1139 bytes} [Tue Jun 7 08:20:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:06.848639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410af7b9-32da-4978-9871-8ff047ef2a22 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:06.870613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:06.924865 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:06.927870 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bc23b-fc76-4543-be8c-70c2e92d20b2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:06.949726 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410af7b9-32da-4978-9871-8ff047ef2a22 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:06.961413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f0879-688b-455e-a9dd-a6513954bff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:06.962131 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f0879-688b-455e-a9dd-a6513954bff3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:06.962418 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f0879-688b-455e-a9dd-a6513954bff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:06.962694 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f0879-688b-455e-a9dd-a6513954bff3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:06.963534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f0879-688b-455e-a9dd-a6513954bff3 None admin] RBAC: Adding query filter params (role.id=a04f2ab5f1234bd6b1dc4c18b4210a89, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=ee42568b059b49a198ed2c01d5efdafd) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:06.963844 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f0879-688b-455e-a9dd-a6513954bff3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:06.965298 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f0879-688b-455e-a9dd-a6513954bff3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:06.971384 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 264/523] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:06 2022] GET /identity/v3/role_assignments?user.id=ee42568b059b49a198ed2c01d5efdafd&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:06.972166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f0879-688b-455e-a9dd-a6513954bff3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:06.986754 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681b48c-404d-408b-b03f-d48090df84b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:06.987435 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681b48c-404d-408b-b03f-d48090df84b5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:06.987716 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681b48c-404d-408b-b03f-d48090df84b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:06.987991 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681b48c-404d-408b-b03f-d48090df84b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:06.988729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681b48c-404d-408b-b03f-d48090df84b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:06.989061 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681b48c-404d-408b-b03f-d48090df84b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:06.990500 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681b48c-404d-408b-b03f-d48090df84b5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:07.005053 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 260/524] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:07.005053 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681b48c-404d-408b-b03f-d48090df84b5 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:07.093067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f0879-688b-455e-a9dd-a6513954bff3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:07.096011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:07.114201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681b48c-404d-408b-b03f-d48090df84b5 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:07.130656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9156e-b3e6-4260-89bb-6e48e8e61a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:07.131258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9156e-b3e6-4260-89bb-6e48e8e61a45 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:07.131545 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9156e-b3e6-4260-89bb-6e48e8e61a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:07.131825 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9156e-b3e6-4260-89bb-6e48e8e61a45 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:07.132681 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd9156e-b3e6-4260-89bb-6e48e8e61a45 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:07.133016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd9156e-b3e6-4260-89bb-6e48e8e61a45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:07.134484 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd9156e-b3e6-4260-89bb-6e48e8e61a45 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:07.150710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f976fe-bfe6-44f1-ba30-b3728967d891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:07.151381 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f976fe-bfe6-44f1-ba30-b3728967d891 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:07.151660 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f976fe-bfe6-44f1-ba30-b3728967d891 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:07.151936 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f976fe-bfe6-44f1-ba30-b3728967d891 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:07.152645 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f976fe-bfe6-44f1-ba30-b3728967d891 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:07.159336 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 265/525] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Jun 7 08:20:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:07.159498 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f976fe-bfe6-44f1-ba30-b3728967d891 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:07.159591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9156e-b3e6-4260-89bb-6e48e8e61a45 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:07.165319 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:07.231244 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 261/526] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jun 7 08:20:07 2022] POST /identity/v3/users => generated 247 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:07.232203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f976fe-bfe6-44f1-ba30-b3728967d891 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:07.282882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9156e-b3e6-4260-89bb-6e48e8e61a45 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:07.323220 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0660e2-51b9-4459-9b7f-b69b4d940a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:07.324244 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0660e2-51b9-4459-9b7f-b69b4d940a5c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:07.324707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0660e2-51b9-4459-9b7f-b69b4d940a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:07.325347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0660e2-51b9-4459-9b7f-b69b4d940a5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:07.361282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f976fe-bfe6-44f1-ba30-b3728967d891 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:07.369093 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:07.407705 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce494b-2301-4987-abdc-2d606b9eb241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:07.408876 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce494b-2301-4987-abdc-2d606b9eb241 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:07.409497 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce494b-2301-4987-abdc-2d606b9eb241 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:07.409996 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce494b-2301-4987-abdc-2d606b9eb241 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:07.418563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce494b-2301-4987-abdc-2d606b9eb241 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:07.420507 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce494b-2301-4987-abdc-2d606b9eb241 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:07.422733 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0660e2-51b9-4459-9b7f-b69b4d940a5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:07.425015 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0660e2-51b9-4459-9b7f-b69b4d940a5c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:07.438090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-7fce494b-2301-4987-abdc-2d606b9eb241 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:07.438090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:07.438090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:07.438090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:07.438090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:07.438090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:07.438090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:07.438090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:07.438090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:07.438090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:07.438090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:07.438090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:07.438090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:07.438090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:07.438090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:07.439010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:07.439010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:07.439010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:07.439010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:07.439010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:07.439010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:07.439010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:07.439010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:07.439010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:07.439010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:07.439010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:07.439010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:07.439010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:07.439010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:07.439010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:07.439010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:07.439010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:07.439844 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:07.439844 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:07.439844 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:07.439844 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:07.439844 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:07.439844 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:07.439844 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:07.439844 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:07.447035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 262/527] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:07.448403 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce494b-2301-4987-abdc-2d606b9eb241 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:07.509061 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:07.548054 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 266/528] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:20:07.549506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0660e2-51b9-4459-9b7f-b69b4d940a5c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:07.567578 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce494b-2301-4987-abdc-2d606b9eb241 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:07.614408 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:07.617123 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9948e-9c7e-4f87-ace7-b5607bea3e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:07.617778 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9948e-9c7e-4f87-ace7-b5607bea3e67 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:07.618060 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9948e-9c7e-4f87-ace7-b5607bea3e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:07.618336 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9948e-9c7e-4f87-ace7-b5607bea3e67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:07.657169 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0660e2-51b9-4459-9b7f-b69b4d940a5c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:07.690936 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce9948e-9c7e-4f87-ace7-b5607bea3e67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:07.692441 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce9948e-9c7e-4f87-ace7-b5607bea3e67 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:07.693436 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ff1f3-de80-412b-97ed-bf85a10a3076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:07.694264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ff1f3-de80-412b-97ed-bf85a10a3076 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:07.695057 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ff1f3-de80-412b-97ed-bf85a10a3076 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:07.695483 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ff1f3-de80-412b-97ed-bf85a10a3076 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:07.696415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ff1f3-de80-412b-97ed-bf85a10a3076 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:07.696869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ff1f3-de80-412b-97ed-bf85a10a3076 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:07.698476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ff1f3-de80-412b-97ed-bf85a10a3076 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:07.717603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 267/529] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:07.718747 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ff1f3-de80-412b-97ed-bf85a10a3076 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:07.765395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:07.801894 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 263/530] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:20:07.802752 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9948e-9c7e-4f87-ace7-b5607bea3e67 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:07.827726 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ff1f3-de80-412b-97ed-bf85a10a3076 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:07.851600 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:07.864103 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0132d21-c053-4207-9f73-3c2ad9c345be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:07.868038 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0132d21-c053-4207-9f73-3c2ad9c345be None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:07.868038 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0132d21-c053-4207-9f73-3c2ad9c345be None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:07.868038 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0132d21-c053-4207-9f73-3c2ad9c345be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:07.880117 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0132d21-c053-4207-9f73-3c2ad9c345be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:07.882194 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0132d21-c053-4207-9f73-3c2ad9c345be None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:07.904875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d0132d21-c053-4207-9f73-3c2ad9c345be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:07.904875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:07.904875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:07.904875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:07.904875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:07.904875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:07.904875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:07.904875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:07.904875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:07.904875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:07.904875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:07.904875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:07.904875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:07.904875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:07.904875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:07.905974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:07.905974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:07.905974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:07.905974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:07.905974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:07.905974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:07.905974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:07.905974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:07.905974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:07.905974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:07.905974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:07.905974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:07.905974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:07.905974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:07.905974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:07.905974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:07.905974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:07.906980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:07.906980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:07.906980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:07.906980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:07.909136 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 268/531] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:07.910316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0132d21-c053-4207-9f73-3c2ad9c345be None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:07.913714 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9948e-9c7e-4f87-ace7-b5607bea3e67 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:07.950854 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6331cf3-6929-48cb-88b2-cd04ab31af2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:07.951806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6331cf3-6929-48cb-88b2-cd04ab31af2c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:07.952223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6331cf3-6929-48cb-88b2-cd04ab31af2c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:07.952636 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6331cf3-6929-48cb-88b2-cd04ab31af2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:07.953681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6331cf3-6929-48cb-88b2-cd04ab31af2c None admin] RBAC: Adding query filter params (role.id=a04f2ab5f1234bd6b1dc4c18b4210a89, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=c4d9b5e3826049f2800f1b9424977d75) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:07.954149 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6331cf3-6929-48cb-88b2-cd04ab31af2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:07.955810 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6331cf3-6929-48cb-88b2-cd04ab31af2c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:07.962253 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 264/532] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:07 2022] GET /identity/v3/role_assignments?user.id=c4d9b5e3826049f2800f1b9424977d75&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:07.963218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6331cf3-6929-48cb-88b2-cd04ab31af2c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:07.983807 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:08.050014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0132d21-c053-4207-9f73-3c2ad9c345be None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:08.095065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6331cf3-6929-48cb-88b2-cd04ab31af2c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:08.096443 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc1ea7-95c8-476b-b578-a34e90e88ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:08.097609 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc1ea7-95c8-476b-b578-a34e90e88ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:08.097895 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc1ea7-95c8-476b-b578-a34e90e88ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:08.098174 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc1ea7-95c8-476b-b578-a34e90e88ad1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:08.112797 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc1ea7-95c8-476b-b578-a34e90e88ad1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:08.114347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc1ea7-95c8-476b-b578-a34e90e88ad1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:08.136949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5ecc1ea7-95c8-476b-b578-a34e90e88ad1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:08.136949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:08.136949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:08.136949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:08.136949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:08.136949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:08.136949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:08.136949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:08.136949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:08.136949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:08.136949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:08.136949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:08.136949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:08.136949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:08.136949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:08.138058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:08.138058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:08.138058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:08.138058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:08.138058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:08.138058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:08.138058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:08.138058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:08.138058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:08.138058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:08.138058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:08.138058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:08.138058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:08.138058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:08.138058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:08.138058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:08.138058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:08.139076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:08.139076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:08.139076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:08.139076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:08.139076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 269/533] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:08.146840 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc1ea7-95c8-476b-b578-a34e90e88ad1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:08.182376 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84a9bc-4d49-41e4-8820-8814314e3986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:08.183132 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84a9bc-4d49-41e4-8820-8814314e3986 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:08.183419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84a9bc-4d49-41e4-8820-8814314e3986 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:08.183746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84a9bc-4d49-41e4-8820-8814314e3986 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:08.184510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc84a9bc-4d49-41e4-8820-8814314e3986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:08.186208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc84a9bc-4d49-41e4-8820-8814314e3986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:08.187755 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc84a9bc-4d49-41e4-8820-8814314e3986 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:08.211029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 265/534] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:08.217616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:08.218526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84a9bc-4d49-41e4-8820-8814314e3986 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:08.351845 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc1ea7-95c8-476b-b578-a34e90e88ad1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:08.369433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:08.395010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84a9bc-4d49-41e4-8820-8814314e3986 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:08.405821 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a67d7-0fda-4638-910f-4bd75d6a67d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:08.406494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a67d7-0fda-4638-910f-4bd75d6a67d4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:08.406778 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a67d7-0fda-4638-910f-4bd75d6a67d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:08.407050 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a67d7-0fda-4638-910f-4bd75d6a67d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:08.407771 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a67d7-0fda-4638-910f-4bd75d6a67d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:08.408074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a67d7-0fda-4638-910f-4bd75d6a67d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:08.409481 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a67d7-0fda-4638-910f-4bd75d6a67d4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:08.421440 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 270/535] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:08.426921 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a67d7-0fda-4638-910f-4bd75d6a67d4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:08.454501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd17853-612d-4356-85a1-f3c472b9c80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:08.455358 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd17853-612d-4356-85a1-f3c472b9c80c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:08.455643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd17853-612d-4356-85a1-f3c472b9c80c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:08.456010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd17853-612d-4356-85a1-f3c472b9c80c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:08.465818 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd17853-612d-4356-85a1-f3c472b9c80c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:08.467831 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd17853-612d-4356-85a1-f3c472b9c80c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:08.482689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-6fd17853-612d-4356-85a1-f3c472b9c80c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:08.482689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:08.482689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:08.482689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:08.482689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:08.482689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:08.482689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:08.482689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:08.482689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:08.482689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:08.482689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:08.482689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:08.482689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:08.482689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:08.482689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:08.483511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:08.483511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:08.483511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:08.483511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:08.483511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:08.483511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:08.483511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:08.483511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:08.483511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:08.483511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:08.483511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:08.483511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:08.483511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:08.483511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:08.483511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:08.483511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:08.483511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:08.484327 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:08.484327 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:08.484327 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:08.484327 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:08.484327 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:08.484327 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:08.484327 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:08.484327 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:08.484327 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 266/536] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:08.492589 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd17853-612d-4356-85a1-f3c472b9c80c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:08.526400 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a67d7-0fda-4638-910f-4bd75d6a67d4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:08.529896 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:08.561133 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a57df-ac10-4e9a-b909-70c41b2bda31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:08.561936 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a57df-ac10-4e9a-b909-70c41b2bda31 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:08.562215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a57df-ac10-4e9a-b909-70c41b2bda31 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:08.562488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a57df-ac10-4e9a-b909-70c41b2bda31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:08.569288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a57df-ac10-4e9a-b909-70c41b2bda31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:08.570741 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a57df-ac10-4e9a-b909-70c41b2bda31 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:08.586621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-608a57df-ac10-4e9a-b909-70c41b2bda31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:08.586621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:08.586621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:08.586621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:08.586621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:08.586621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:08.586621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:08.586621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:08.586621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:08.586621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:08.586621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:08.586621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:08.586621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:08.586621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:08.586621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:08.588540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:08.588540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:08.588540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:08.588540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:08.588540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:08.588540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:08.588540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:08.588540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:08.588540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:08.588540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:08.588540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:08.588540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:08.588540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:08.588540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:08.588540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:08.588540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:08.588540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:08.590609 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:08.590609 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:08.590609 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:08.590609 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:08.590609 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:08.590609 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:08.590609 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:08.590609 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:08.590609 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 271/537] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:08.601026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a57df-ac10-4e9a-b909-70c41b2bda31 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:08.639437 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd17853-612d-4356-85a1-f3c472b9c80c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:08.679866 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea7716-ec5d-40c1-8d29-2a8699dd5ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:08.680597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea7716-ec5d-40c1-8d29-2a8699dd5ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:08.680884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea7716-ec5d-40c1-8d29-2a8699dd5ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:08.681190 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea7716-ec5d-40c1-8d29-2a8699dd5ea2 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:08.681993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea7716-ec5d-40c1-8d29-2a8699dd5ea2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:08.682309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea7716-ec5d-40c1-8d29-2a8699dd5ea2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:08.683775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea7716-ec5d-40c1-8d29-2a8699dd5ea2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:08.690294 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:08.712141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 267/538] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:08.720217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea7716-ec5d-40c1-8d29-2a8699dd5ea2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:08.784973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a57df-ac10-4e9a-b909-70c41b2bda31 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:08.834268 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea7716-ec5d-40c1-8d29-2a8699dd5ea2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:08.849543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb2fc2-396a-486d-9a70-1f25e2d273de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:08.850206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb2fc2-396a-486d-9a70-1f25e2d273de None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:08.850541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb2fc2-396a-486d-9a70-1f25e2d273de None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:08.850817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb2fc2-396a-486d-9a70-1f25e2d273de None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:08.851752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb2fc2-396a-486d-9a70-1f25e2d273de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:08.852161 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb2fc2-396a-486d-9a70-1f25e2d273de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:08.853634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb2fc2-396a-486d-9a70-1f25e2d273de None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:08.867097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 272/539] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:08.869583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce3cd4-f051-43c3-bfcc-b138a9c279ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:08.870234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce3cd4-f051-43c3-bfcc-b138a9c279ae None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:08.870515 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce3cd4-f051-43c3-bfcc-b138a9c279ae None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:08.870790 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce3cd4-f051-43c3-bfcc-b138a9c279ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:08.871930 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce3cd4-f051-43c3-bfcc-b138a9c279ae None admin] RBAC: Adding query filter params (role.id=a04f2ab5f1234bd6b1dc4c18b4210a89, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=4a87f8bb06ee4a3ba7017422f4f1943e) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:08.872240 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce3cd4-f051-43c3-bfcc-b138a9c279ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:08.874257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce3cd4-f051-43c3-bfcc-b138a9c279ae None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:08.874707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb2fc2-396a-486d-9a70-1f25e2d273de None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:08.880850 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 268/540] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:08 2022] GET /identity/v3/role_assignments?user.id=4a87f8bb06ee4a3ba7017422f4f1943e&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:08.887795 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:09.042103 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb2fc2-396a-486d-9a70-1f25e2d273de None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:09.082313 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cfa34-ab77-42b0-8980-daf94125c6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:09.082974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cfa34-ab77-42b0-8980-daf94125c6be None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:09.083253 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cfa34-ab77-42b0-8980-daf94125c6be None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:09.083523 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cfa34-ab77-42b0-8980-daf94125c6be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:09.084361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6cfa34-ab77-42b0-8980-daf94125c6be None admin] RBAC: Adding query filter params (role.id=a04f2ab5f1234bd6b1dc4c18b4210a89, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=85ea2962e7094f8c878d95680764c4e6) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:09.084671 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6cfa34-ab77-42b0-8980-daf94125c6be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:09.086145 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6cfa34-ab77-42b0-8980-daf94125c6be None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:09.092368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 273/541] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:08 2022] GET /identity/v3/role_assignments?user.id=85ea2962e7094f8c878d95680764c4e6&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:09.561171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0a3c2-5546-4f12-932b-4af6da9b4cac None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:09.561689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0a3c2-5546-4f12-932b-4af6da9b4cac None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:09.561793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0a3c2-5546-4f12-932b-4af6da9b4cac None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:09.563376 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 269/542] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:09.568640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6891215-c31d-4c0c-a64b-b95b8e846d57 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:09.569079 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6891215-c31d-4c0c-a64b-b95b8e846d57 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:09.569243 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6891215-c31d-4c0c-a64b-b95b8e846d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:09.594019 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f6891215-c31d-4c0c-a64b-b95b8e846d57 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:09.595225 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f6891215-c31d-4c0c-a64b-b95b8e846d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:09.666654 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:09.672678 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 274/543] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:09.695477 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0a3c2-5546-4f12-932b-4af6da9b4cac None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:09.763110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:09.809172 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0a3c2-5546-4f12-932b-4af6da9b4cac None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:09.844088 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1de3c-c2f1-4426-9639-ee6889558ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:09.844808 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1de3c-c2f1-4426-9639-ee6889558ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:09.845122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1de3c-c2f1-4426-9639-ee6889558ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:09.845400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1de3c-c2f1-4426-9639-ee6889558ed4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:09.856104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1de3c-c2f1-4426-9639-ee6889558ed4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:09.864595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-94c1de3c-c2f1-4426-9639-ee6889558ed4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:09.864595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:09.864595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:09.864595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:09.864595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:09.864595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:09.864595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:09.864595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:09.864595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:09.864595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:09.864595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:09.864595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:09.864595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:09.864595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:09.864595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:09.864595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:09.865502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:09.865502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:09.865502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:09.865502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:09.865502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:09.865502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:09.865502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:09.865502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:09.865502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:09.865502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:09.865502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:09.865502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:09.865502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:09.865502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:09.865502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:09.865502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:09.866285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:09.866285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:09.866285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:09.866285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:09.866285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:09.866285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:09.866285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:09.866285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:09.866285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:09.866285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:09.866285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:09.866285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:09.866285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 270/544] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:09.871919 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6891215-c31d-4c0c-a64b-b95b8e846d57 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:09.978435 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6891215-c31d-4c0c-a64b-b95b8e846d57 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:10.025353 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c725d00-0575-4a9e-babf-afbba4995413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:10.026085 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c725d00-0575-4a9e-babf-afbba4995413 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:10.026373 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c725d00-0575-4a9e-babf-afbba4995413 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:10.026653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c725d00-0575-4a9e-babf-afbba4995413 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:10.027403 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c725d00-0575-4a9e-babf-afbba4995413 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:10.027710 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c725d00-0575-4a9e-babf-afbba4995413 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:10.029665 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c725d00-0575-4a9e-babf-afbba4995413 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:10.038709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 275/545] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:20:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:10.044320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1de3c-c2f1-4426-9639-ee6889558ed4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:10.150420 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1de3c-c2f1-4426-9639-ee6889558ed4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:10.174965 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:10.201908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411c893-b104-4201-a0cb-4fc7bc09e16e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:10.202078 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2c623-4747-4bb1-86ad-5fadadc4c1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:10.202278 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411c893-b104-4201-a0cb-4fc7bc09e16e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:10.205712 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411c893-b104-4201-a0cb-4fc7bc09e16e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:10.205808 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2c623-4747-4bb1-86ad-5fadadc4c1b4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:10.205808 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2c623-4747-4bb1-86ad-5fadadc4c1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:10.205808 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2c623-4747-4bb1-86ad-5fadadc4c1b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:10.206005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 276/546] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:10.209816 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4dd48-a888-4dcb-a182-12516552db65 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:10.210320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4dd48-a888-4dcb-a182-12516552db65 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:10.210610 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4dd48-a888-4dcb-a182-12516552db65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:10.220727 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2c623-4747-4bb1-86ad-5fadadc4c1b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:10.222218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2c623-4747-4bb1-86ad-5fadadc4c1b4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:10.230275 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d4a4dd48-a888-4dcb-a182-12516552db65 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:10.231877 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d4a4dd48-a888-4dcb-a182-12516552db65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:10.250612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-50c2c623-4747-4bb1-86ad-5fadadc4c1b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:10.250612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:10.250612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:10.250612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:10.250612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:10.250612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:10.250612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:10.250612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:10.250612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:10.250612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:10.250612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:10.250612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:10.250612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:10.250612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:10.250612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:10.251458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:10.251458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:10.251458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:10.251458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:10.251458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:10.251458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:10.251458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:10.251458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:10.251458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:10.251458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:10.251458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:10.251458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:10.251458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:10.251458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:10.251458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:10.251458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:10.251458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:10.252295 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:10.252295 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:10.252295 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:10.252295 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:10.253475 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 271/547] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:10.260472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2c623-4747-4bb1-86ad-5fadadc4c1b4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:10.388072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:10.407922 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 277/548] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:10.478651 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4dd48-a888-4dcb-a182-12516552db65 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:10.515490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2c623-4747-4bb1-86ad-5fadadc4c1b4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:10.586991 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea839c-82a4-4ef8-92d6-dd8524cc13bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:10.587759 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea839c-82a4-4ef8-92d6-dd8524cc13bb None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:10.587942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea839c-82a4-4ef8-92d6-dd8524cc13bb None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:10.588217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea839c-82a4-4ef8-92d6-dd8524cc13bb None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:10.588948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea839c-82a4-4ef8-92d6-dd8524cc13bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:10.589281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea839c-82a4-4ef8-92d6-dd8524cc13bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:10.590662 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea839c-82a4-4ef8-92d6-dd8524cc13bb None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:10.612217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 272/549] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:10.617730 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea839c-82a4-4ef8-92d6-dd8524cc13bb None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:10.637236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4dd48-a888-4dcb-a182-12516552db65 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:10.670787 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:10.709746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2d05f-a9a8-4409-819b-729009015901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:10.711083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2d05f-a9a8-4409-819b-729009015901 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:10.711692 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2d05f-a9a8-4409-819b-729009015901 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:10.713114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2d05f-a9a8-4409-819b-729009015901 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:10.732795 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2d05f-a9a8-4409-819b-729009015901 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:10.740819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1cd2d05f-a9a8-4409-819b-729009015901 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:10.740819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:10.740819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:10.740819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:10.740819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:10.740819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:10.740819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:10.740819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:10.740819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:10.740819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:10.740819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:10.740819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:10.740819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:10.740819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:10.740819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:10.740819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:10.741756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:10.741756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:10.741756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:10.741756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:10.741756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:10.741756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:10.741756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:10.741756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:10.741756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:10.741756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:10.741756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:10.741756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:10.741756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:10.741756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:10.741756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:10.741756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:10.742574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:10.742574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:10.742574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:10.742574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:10.742574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:10.742574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:10.742574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:10.742574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:10.742574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:10.742574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:10.742574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:10.742574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:10.745174 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 278/550] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:10.746602 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a47b4d-9315-4742-947e-4324ff04fcb2 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:10.747085 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a47b4d-9315-4742-947e-4324ff04fcb2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:10.747617 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a47b4d-9315-4742-947e-4324ff04fcb2 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:10.749635 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 279/551] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:10.752491 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a47b4d-9315-4742-947e-4324ff04fcb2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:10.762989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea839c-82a4-4ef8-92d6-dd8524cc13bb None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:10.798260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc0c74-ea4d-4832-bc49-93353e788054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:10.799468 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc0c74-ea4d-4832-bc49-93353e788054 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:10.799678 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc0c74-ea4d-4832-bc49-93353e788054 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:10.799935 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc0c74-ea4d-4832-bc49-93353e788054 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:10.807382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc0c74-ea4d-4832-bc49-93353e788054 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:10.808847 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc0c74-ea4d-4832-bc49-93353e788054 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:10.810107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:10.824395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-1bfc0c74-ea4d-4832-bc49-93353e788054 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:20:10.824395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:10.824395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:10.824395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:10.824395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:10.824395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:10.824395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:10.824395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:10.824395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:10.824395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:10.824395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:10.824395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:10.824395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:10.824395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:10.824395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:10.824395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:10.825352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:10.825352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:10.825352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:10.825352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:10.825352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:10.825352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:10.825352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:10.825352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:10.825352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:10.825352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:10.825352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:10.825352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:10.825352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:10.825352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:10.825352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:10.825352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:10.825352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:10.825352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:10.826281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:10.826281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:10.826281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:10.826281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:10.826281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:10.826281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:10.826281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:10.826281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:10.826281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:20:10.826281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:10.826281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 273/552] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Jun 7 08:20:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:10.827314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9f5d8-46b2-485b-b11a-e5bfb0109cc1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:10.827410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9f5d8-46b2-485b-b11a-e5bfb0109cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:10.827643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9f5d8-46b2-485b-b11a-e5bfb0109cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:10.848685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-2cc9f5d8-46b2-485b-b11a-e5bfb0109cc1 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:10.849748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2cc9f5d8-46b2-485b-b11a-e5bfb0109cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:10.865684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a47b4d-9315-4742-947e-4324ff04fcb2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:10.901365 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb795fe-46e6-451e-89fb-ae5592a6ac22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:10.902358 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb795fe-46e6-451e-89fb-ae5592a6ac22 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:10.902840 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb795fe-46e6-451e-89fb-ae5592a6ac22 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:10.903412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb795fe-46e6-451e-89fb-ae5592a6ac22 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:10.904406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb795fe-46e6-451e-89fb-ae5592a6ac22 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:10.904929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb795fe-46e6-451e-89fb-ae5592a6ac22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:10.906516 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb795fe-46e6-451e-89fb-ae5592a6ac22 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:10.919522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 280/553] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:20:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:10.921657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb795fe-46e6-451e-89fb-ae5592a6ac22 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:10.923090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 274/554] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:10.924264 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9f5d8-46b2-485b-b11a-e5bfb0109cc1 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:10.946763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:11.027766 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb795fe-46e6-451e-89fb-ae5592a6ac22 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:11.032378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9f5d8-46b2-485b-b11a-e5bfb0109cc1 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:11.043532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:11.073493 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ab2cf-89bc-46b1-91dd-9d34ecb8e767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:11.073698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327d5a5-6e68-4ed3-950c-23840c45f620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:11.074340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ab2cf-89bc-46b1-91dd-9d34ecb8e767 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:11.074449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327d5a5-6e68-4ed3-950c-23840c45f620 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:11.074630 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ab2cf-89bc-46b1-91dd-9d34ecb8e767 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:11.074725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327d5a5-6e68-4ed3-950c-23840c45f620 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:11.074889 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ab2cf-89bc-46b1-91dd-9d34ecb8e767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:11.074986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327d5a5-6e68-4ed3-950c-23840c45f620 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:11.075744 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327d5a5-6e68-4ed3-950c-23840c45f620 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:11.076085 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327d5a5-6e68-4ed3-950c-23840c45f620 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:11.078233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327d5a5-6e68-4ed3-950c-23840c45f620 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:11.086827 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ab2cf-89bc-46b1-91dd-9d34ecb8e767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:11.088612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ab2cf-89bc-46b1-91dd-9d34ecb8e767 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:11.100545 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 281/555] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Jun 7 08:20:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:11.101616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327d5a5-6e68-4ed3-950c-23840c45f620 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:11.116168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-c27ab2cf-89bc-46b1-91dd-9d34ecb8e767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:11.116168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:11.116168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:11.116168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:11.116168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:11.116168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:11.116168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:11.116168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:11.116168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:11.116168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:11.116168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:11.116168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:11.116168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:11.116168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:11.116168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:11.117110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:11.117110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:11.117110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:11.117110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:11.117110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:11.117110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:11.117110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:11.117110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:11.117110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:11.117110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:11.117110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:11.117110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:11.117110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:11.117110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:11.117110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:11.117110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:11.117110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:11.118237 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:11.118237 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:11.118237 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:11.118237 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:11.120909 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 275/556] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:11.120909 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ab2cf-89bc-46b1-91dd-9d34ecb8e767 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:11.202256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:11.230961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327d5a5-6e68-4ed3-950c-23840c45f620 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:11.236419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ab2cf-89bc-46b1-91dd-9d34ecb8e767 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:11.269190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083deb7-df88-42a2-b366-a122e246a9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:11.270205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083deb7-df88-42a2-b366-a122e246a9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:11.270557 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083deb7-df88-42a2-b366-a122e246a9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:11.270895 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083deb7-df88-42a2-b366-a122e246a9f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:11.273636 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea621ad-e8a5-4ff1-8f88-359c35ce213c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:11.274832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea621ad-e8a5-4ff1-8f88-359c35ce213c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:11.275265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea621ad-e8a5-4ff1-8f88-359c35ce213c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:11.275691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea621ad-e8a5-4ff1-8f88-359c35ce213c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:11.286289 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083deb7-df88-42a2-b366-a122e246a9f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:11.288562 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea621ad-e8a5-4ff1-8f88-359c35ce213c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:11.290275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea621ad-e8a5-4ff1-8f88-359c35ce213c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:11.297096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d083deb7-df88-42a2-b366-a122e246a9f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:11.297096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:11.297096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:11.297096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:11.297096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:11.297096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:11.297096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:11.297096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:11.297096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:11.297096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:11.297096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:11.297096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:11.297096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:11.297096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:11.297096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:11.297096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:11.298613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:11.298613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:11.298613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:11.298613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:11.298613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:11.298613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:11.298613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:11.298613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:11.298613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:11.298613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:11.298613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:11.298613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:11.298613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:11.298613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:11.298613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:11.298613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:11.300413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:11.300413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:11.300413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:11.300413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:11.300413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:11.300413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:11.300413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:11.300413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:11.300413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:11.300413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:11.300413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:11.300413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:11.300413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 282/557] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:11.301245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083deb7-df88-42a2-b366-a122e246a9f9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:11.316462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-4ea621ad-e8a5-4ff1-8f88-359c35ce213c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:11.316462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:11.316462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:11.316462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:11.316462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:11.316462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:11.316462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:11.316462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:11.316462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:11.316462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:11.316462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:11.316462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:11.316462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:11.316462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:11.316462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:11.317683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:11.317683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:11.317683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:11.317683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:11.317683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:11.317683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:11.317683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:11.317683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:11.317683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:11.317683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:11.317683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:11.317683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:11.317683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:11.317683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:11.317683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:11.317683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:11.317683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:11.318586 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:11.318586 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:11.318586 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:11.318586 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:11.320622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 276/558] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:11.321769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea621ad-e8a5-4ff1-8f88-359c35ce213c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:11.347751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:11.439324 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083deb7-df88-42a2-b366-a122e246a9f9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:11.453760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea621ad-e8a5-4ff1-8f88-359c35ce213c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:11.480853 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3e923-7392-4d03-978f-8a5ff5cc2a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:11.482025 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3e923-7392-4d03-978f-8a5ff5cc2a3e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:11.482705 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3e923-7392-4d03-978f-8a5ff5cc2a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:11.483198 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3e923-7392-4d03-978f-8a5ff5cc2a3e None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:11.484214 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3e923-7392-4d03-978f-8a5ff5cc2a3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:11.484774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3e923-7392-4d03-978f-8a5ff5cc2a3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:11.485509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:11.487953 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3e923-7392-4d03-978f-8a5ff5cc2a3e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:11.498380 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded383c-ab29-496d-9ac5-88db3c273945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:11.499334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded383c-ab29-496d-9ac5-88db3c273945 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:11.499625 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded383c-ab29-496d-9ac5-88db3c273945 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:11.499901 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded383c-ab29-496d-9ac5-88db3c273945 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:11.500656 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded383c-ab29-496d-9ac5-88db3c273945 None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:11.500964 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded383c-ab29-496d-9ac5-88db3c273945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:11.502232 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 283/559] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:11.502501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded383c-ab29-496d-9ac5-88db3c273945 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:11.503843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3e923-7392-4d03-978f-8a5ff5cc2a3e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:11.512705 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 277/560] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:20:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:11.514104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded383c-ab29-496d-9ac5-88db3c273945 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:11.623114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:11.631801 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded383c-ab29-496d-9ac5-88db3c273945 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:11.633260 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3e923-7392-4d03-978f-8a5ff5cc2a3e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:11.669881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969e116-7dd8-48d2-84ca-ac4f19235cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:11.670128 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81cdf10-fc5d-4dff-98ca-7aa54f91c8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:11.670580 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81cdf10-fc5d-4dff-98ca-7aa54f91c8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:11.670658 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969e116-7dd8-48d2-84ca-ac4f19235cde None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:11.670914 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81cdf10-fc5d-4dff-98ca-7aa54f91c8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:11.671025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969e116-7dd8-48d2-84ca-ac4f19235cde None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:11.671111 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81cdf10-fc5d-4dff-98ca-7aa54f91c8f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:11.671184 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969e116-7dd8-48d2-84ca-ac4f19235cde None admin] PATH_INFO: `/v3/users/swift` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:11.671795 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81cdf10-fc5d-4dff-98ca-7aa54f91c8f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:11.672106 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81cdf10-fc5d-4dff-98ca-7aa54f91c8f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:11.673516 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81cdf10-fc5d-4dff-98ca-7aa54f91c8f7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:11.679594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969e116-7dd8-48d2-84ca-ac4f19235cde None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:11.681190 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969e116-7dd8-48d2-84ca-ac4f19235cde None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:11.689354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 284/561] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:11.689354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81cdf10-fc5d-4dff-98ca-7aa54f91c8f7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:11.698597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-9969e116-7dd8-48d2-84ca-ac4f19235cde None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:20:11.698597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:11.698597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:11.698597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:11.698597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:11.698597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:11.698597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:11.698597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:11.698597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:11.698597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:11.698597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:11.698597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:11.698597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:11.698597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:11.698597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:11.698597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:11.699644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:11.699644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:11.699644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:11.699644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:11.699644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:11.699644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:11.699644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:11.699644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:11.699644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:11.699644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:11.699644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:11.699644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:11.699644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:11.699644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:11.699644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:11.699644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:11.699644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:11.699644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:11.700600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:11.700600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:11.700600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:11.700600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:11.700600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:11.700600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:11.700600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:11.700600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:11.700600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:20:11.700600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:11.700600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 278/562] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:20:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:11.701768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42e7c2-b6ac-47ae-9170-644435d3cab6 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:11.702328 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42e7c2-b6ac-47ae-9170-644435d3cab6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:11.702328 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42e7c2-b6ac-47ae-9170-644435d3cab6 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:11.705308 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 279/563] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:11.706296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42e7c2-b6ac-47ae-9170-644435d3cab6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:11.754684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:11.793293 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81cdf10-fc5d-4dff-98ca-7aa54f91c8f7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:11.806816 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42e7c2-b6ac-47ae-9170-644435d3cab6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:11.827236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a18f9-8a3d-4a75-b915-c4f966f40446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:11.828026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a18f9-8a3d-4a75-b915-c4f966f40446 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:11.828307 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a18f9-8a3d-4a75-b915-c4f966f40446 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:11.828583 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a18f9-8a3d-4a75-b915-c4f966f40446 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:11.838985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a18f9-8a3d-4a75-b915-c4f966f40446 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:11.839506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715d4a2-e5df-49e2-9576-06c440f26f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:11.840361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715d4a2-e5df-49e2-9576-06c440f26f41 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:11.840453 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a18f9-8a3d-4a75-b915-c4f966f40446 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:11.840772 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715d4a2-e5df-49e2-9576-06c440f26f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:11.841083 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715d4a2-e5df-49e2-9576-06c440f26f41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:11.847640 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e715d4a2-e5df-49e2-9576-06c440f26f41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:11.849095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e715d4a2-e5df-49e2-9576-06c440f26f41 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:11.857205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-ce0a18f9-8a3d-4a75-b915-c4f966f40446 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:11.857205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:11.857205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:11.857205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:11.857205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:11.857205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:11.857205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:11.857205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:11.857205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:11.857205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:11.857205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:11.857205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:11.857205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:11.857205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:11.857205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:11.858238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:11.858238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:11.858238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:11.858238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:11.858238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:11.858238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:11.858238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:11.858238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:11.858238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:11.858238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:11.858238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:11.858238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:11.858238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:11.858238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:11.858238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:11.858238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:11.858238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:11.859146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:11.859146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:11.859146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:11.859146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:11.859146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 285/564] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:11.859429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a18f9-8a3d-4a75-b915-c4f966f40446 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:11.863159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-e715d4a2-e5df-49e2-9576-06c440f26f41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:11.863159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:11.863159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:11.863159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:11.863159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:11.863159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:11.863159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:11.863159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:11.863159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:11.863159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:11.863159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:11.863159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:11.863159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:11.863159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:11.863159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:11.864204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:11.864204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:11.864204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:11.864204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:11.864204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:11.864204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:11.864204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:11.864204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:11.864204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:11.864204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:11.864204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:11.864204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:11.864204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:11.864204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:11.864204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:11.864204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:11.864204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:11.865129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:11.865129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:11.865129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:11.865129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:11.865129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:11.865129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:11.865129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:11.865129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:11.865129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 280/565] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:11.865629 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:11.865699 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad80847-bad5-43fe-8f76-3154ebee016b None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:11.865980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad80847-bad5-43fe-8f76-3154ebee016b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:11.866342 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad80847-bad5-43fe-8f76-3154ebee016b None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:11.867444 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 281/566] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:11.868732 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34de7a8-3dbe-4710-81c6-af9e6faeead6 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:11.868911 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34de7a8-3dbe-4710-81c6-af9e6faeead6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:11.869250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34de7a8-3dbe-4710-81c6-af9e6faeead6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:11.886330 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b34de7a8-3dbe-4710-81c6-af9e6faeead6 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:11.887519 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b34de7a8-3dbe-4710-81c6-af9e6faeead6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:11.953037 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 282/567] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:11.954035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34de7a8-3dbe-4710-81c6-af9e6faeead6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:11.963689 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a18f9-8a3d-4a75-b915-c4f966f40446 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:11.994841 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:11.997023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4174b-d253-42e6-846f-bcce83331bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:11.998238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4174b-d253-42e6-846f-bcce83331bab None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:11.998519 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4174b-d253-42e6-846f-bcce83331bab None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:11.998795 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4174b-d253-42e6-846f-bcce83331bab None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:11.999625 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4174b-d253-42e6-846f-bcce83331bab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:11.999937 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4174b-d253-42e6-846f-bcce83331bab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:12.001406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4174b-d253-42e6-846f-bcce83331bab None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:12.018612 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 286/568] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Jun 7 08:20:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:12.019401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4174b-d253-42e6-846f-bcce83331bab None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:12.062419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34de7a8-3dbe-4710-81c6-af9e6faeead6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:12.079074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:12.095665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a4dcb-9275-4f53-8211-54e3c37b4f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:12.096329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a4dcb-9275-4f53-8211-54e3c37b4f50 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:12.096617 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a4dcb-9275-4f53-8211-54e3c37b4f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:12.099220 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a4dcb-9275-4f53-8211-54e3c37b4f50 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:12.099220 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a4dcb-9275-4f53-8211-54e3c37b4f50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:12.099220 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a4dcb-9275-4f53-8211-54e3c37b4f50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:12.099440 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a4dcb-9275-4f53-8211-54e3c37b4f50 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:12.111070 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 283/569] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:12.112248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33562a-f177-447f-9e3d-dcb59323b9a4 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:12.112536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33562a-f177-447f-9e3d-dcb59323b9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:12.112815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33562a-f177-447f-9e3d-dcb59323b9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:12.121224 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4174b-d253-42e6-846f-bcce83331bab None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:12.129889 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-0d33562a-f177-447f-9e3d-dcb59323b9a4 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:12.131067 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0d33562a-f177-447f-9e3d-dcb59323b9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:12.154256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fcd885-fc27-4282-b5e6-741d7a44ac6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:12.154907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcd885-fc27-4282-b5e6-741d7a44ac6a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:12.155194 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcd885-fc27-4282-b5e6-741d7a44ac6a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:12.155468 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcd885-fc27-4282-b5e6-741d7a44ac6a None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:12.156280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fcd885-fc27-4282-b5e6-741d7a44ac6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:12.156618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fcd885-fc27-4282-b5e6-741d7a44ac6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:12.158075 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fcd885-fc27-4282-b5e6-741d7a44ac6a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:12.170323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 287/570] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:12.171130 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fcd885-fc27-4282-b5e6-741d7a44ac6a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:12.199516 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 284/571] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:12.200360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33562a-f177-447f-9e3d-dcb59323b9a4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:12.231309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:12.272755 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fcd885-fc27-4282-b5e6-741d7a44ac6a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:12.295572 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:12.304948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33562a-f177-447f-9e3d-dcb59323b9a4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:12.307047 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2769576-4c53-4248-a5c1-c7e82cde2312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:12.307837 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2769576-4c53-4248-a5c1-c7e82cde2312 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:12.308115 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2769576-4c53-4248-a5c1-c7e82cde2312 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:12.308390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2769576-4c53-4248-a5c1-c7e82cde2312 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:12.318573 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2769576-4c53-4248-a5c1-c7e82cde2312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:12.326166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d2769576-4c53-4248-a5c1-c7e82cde2312 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:12.326166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:12.326166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:12.326166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:12.326166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:12.326166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:12.326166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:12.326166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:12.326166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:12.326166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:12.326166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:12.326166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:12.326166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:12.326166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:12.326166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:12.326166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:12.327272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:12.327272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:12.327272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:12.327272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:12.327272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:12.327272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:12.327272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:12.327272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:12.327272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:12.327272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:12.327272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:12.327272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:12.327272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:12.327272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:12.327272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:12.327272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:12.328192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:12.328192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:12.328192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:12.328192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:12.328192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:12.328192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:12.328192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:12.328192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:12.328192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:12.328192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:12.328192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:12.328192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:12.328192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 288/572] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:12.328865 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2769576-4c53-4248-a5c1-c7e82cde2312 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:12.340662 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee113f-73cb-44f8-a851-b1281db028e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:12.341349 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee113f-73cb-44f8-a851-b1281db028e2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:12.341663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee113f-73cb-44f8-a851-b1281db028e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:12.341936 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee113f-73cb-44f8-a851-b1281db028e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:12.407341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee113f-73cb-44f8-a851-b1281db028e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:12.408787 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee113f-73cb-44f8-a851-b1281db028e2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:12.427062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2769576-4c53-4248-a5c1-c7e82cde2312 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:12.435118 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:12.460110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece45f7-6aa3-429d-9d6e-dc65f67bf266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:12.460971 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece45f7-6aa3-429d-9d6e-dc65f67bf266 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:12.461287 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece45f7-6aa3-429d-9d6e-dc65f67bf266 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:12.461556 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece45f7-6aa3-429d-9d6e-dc65f67bf266 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:12.469584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece45f7-6aa3-429d-9d6e-dc65f67bf266 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:12.471011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece45f7-6aa3-429d-9d6e-dc65f67bf266 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:12.486078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9ece45f7-6aa3-429d-9d6e-dc65f67bf266 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 08:20:12.486078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:12.486078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:12.486078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:12.486078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:12.486078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:12.486078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:12.486078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:12.486078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:12.486078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:12.486078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:12.486078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:12.486078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:12.486078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:12.486078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:12.486078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:12.486989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:12.486989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:12.486989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:12.486989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:12.486989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:12.486989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:12.486989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:12.486989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:12.486989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:12.486989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:12.486989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:12.486989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:12.486989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:12.486989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:12.486989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:12.486989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:12.486989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:12.486989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:12.487924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:12.487924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:12.487924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:12.487924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:12.487924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:12.487924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:12.487924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:12.487924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:12.487924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 08:20:12.487924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:12.487924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 289/573] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:12.487924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece45f7-6aa3-429d-9d6e-dc65f67bf266 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:12.510813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 285/574] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:20:12.511823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee113f-73cb-44f8-a851-b1281db028e2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:12.531896 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:12.586345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece45f7-6aa3-429d-9d6e-dc65f67bf266 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:12.606518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee113f-73cb-44f8-a851-b1281db028e2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:12.619185 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b45071-3b49-43ae-8493-6ab51615300d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:12.620062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b45071-3b49-43ae-8493-6ab51615300d None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:12.620340 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b45071-3b49-43ae-8493-6ab51615300d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:12.620617 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b45071-3b49-43ae-8493-6ab51615300d None admin] PATH_INFO: `/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/ee42568b059b49a198ed2c01d5efdafd/roles/a04f2ab5f1234bd6b1dc4c18b4210a89` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:12.632299 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:12.634621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b45071-3b49-43ae-8493-6ab51615300d None admin] RBAC: Authorizing `identity:create_grant(project_id=afc8781e5f864d4a9374300ee9b5193b, user_id=ee42568b059b49a198ed2c01d5efdafd, role_id=a04f2ab5f1234bd6b1dc4c18b4210a89)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:12.636044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b45071-3b49-43ae-8493-6ab51615300d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:12.641476 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779872e9-e62c-4b6d-93e1-83beee4601a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:12.642251 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779872e9-e62c-4b6d-93e1-83beee4601a7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:12.642529 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779872e9-e62c-4b6d-93e1-83beee4601a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:12.642802 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779872e9-e62c-4b6d-93e1-83beee4601a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:12.652340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779872e9-e62c-4b6d-93e1-83beee4601a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:12.660432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-779872e9-e62c-4b6d-93e1-83beee4601a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:12.660432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:12.660432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:12.660432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:12.660432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:12.660432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:12.660432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:12.660432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:12.660432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:12.660432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:12.660432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:12.660432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:12.660432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:12.660432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:12.660432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:12.660432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:12.661536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:12.661536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:12.661536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:12.661536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:12.661536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:12.661536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:12.661536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:12.661536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:12.661536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:12.661536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:12.661536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:12.661536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:12.661536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:12.661536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:12.661536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:12.661536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:12.662552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:12.662552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:12.662552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:12.662552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:12.662552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:12.662552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:12.662552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:12.662552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:12.662552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:12.662552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:12.662552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:12.662552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:12.662552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 286/575] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:12.662552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779872e9-e62c-4b6d-93e1-83beee4601a7 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:12.722129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 290/576] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:20:12 2022] PUT /identity/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/ee42568b059b49a198ed2c01d5efdafd/roles/a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:20:12.722947 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b45071-3b49-43ae-8493-6ab51615300d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:12.775246 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779872e9-e62c-4b6d-93e1-83beee4601a7 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:12.812399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b5635-b60c-45d8-ae5d-58a34ccbac09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:12.813110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b5635-b60c-45d8-ae5d-58a34ccbac09 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:12.813397 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b5635-b60c-45d8-ae5d-58a34ccbac09 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:12.813674 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b5635-b60c-45d8-ae5d-58a34ccbac09 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:12.814431 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b5635-b60c-45d8-ae5d-58a34ccbac09 None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:12.814755 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b5635-b60c-45d8-ae5d-58a34ccbac09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:12.816169 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b5635-b60c-45d8-ae5d-58a34ccbac09 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:12.829774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 287/577] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:20:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:12.829774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b5635-b60c-45d8-ae5d-58a34ccbac09 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:12.831188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:12.839880 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b45071-3b49-43ae-8493-6ab51615300d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:12.881209 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053839d-ed67-4819-bfe7-716b0eb7d394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:12.882377 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053839d-ed67-4819-bfe7-716b0eb7d394 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:12.882670 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053839d-ed67-4819-bfe7-716b0eb7d394 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:12.882943 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053839d-ed67-4819-bfe7-716b0eb7d394 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:12.884077 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053839d-ed67-4819-bfe7-716b0eb7d394 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:12.884386 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053839d-ed67-4819-bfe7-716b0eb7d394 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:12.884874 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:12.886332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053839d-ed67-4819-bfe7-716b0eb7d394 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:12.909548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 291/578] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Jun 7 08:20:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:12.910828 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053839d-ed67-4819-bfe7-716b0eb7d394 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:12.944018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b5635-b60c-45d8-ae5d-58a34ccbac09 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:12.978987 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9873af2-28a1-4ab9-a010-0731f10d7308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:12.979777 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9873af2-28a1-4ab9-a010-0731f10d7308 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:12.980063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9873af2-28a1-4ab9-a010-0731f10d7308 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:12.980340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9873af2-28a1-4ab9-a010-0731f10d7308 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:12.991716 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9873af2-28a1-4ab9-a010-0731f10d7308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:12.993217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9873af2-28a1-4ab9-a010-0731f10d7308 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:13.010236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-f9873af2-28a1-4ab9-a010-0731f10d7308 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:13.010236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:13.010236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:13.010236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:13.010236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:13.010236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:13.010236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:13.010236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:13.010236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:13.010236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:13.010236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:13.010236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:13.010236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:13.010236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:13.010236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:13.011176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:13.011176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:13.011176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:13.011176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:13.011176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:13.011176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:13.011176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:13.011176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:13.011176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:13.011176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:13.011176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:13.011176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:13.011176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:13.011176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:13.011176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:13.011176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:13.011176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:13.012063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:13.012063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:13.012063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:13.012063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:13.012063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 288/579] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:13.012345 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9873af2-28a1-4ab9-a010-0731f10d7308 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:13.029999 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053839d-ed67-4819-bfe7-716b0eb7d394 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:13.066196 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a36e2a-503c-47ca-b069-96b11ae767c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:13.066866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a36e2a-503c-47ca-b069-96b11ae767c6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:13.067150 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a36e2a-503c-47ca-b069-96b11ae767c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:13.067429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a36e2a-503c-47ca-b069-96b11ae767c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:13.068166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a36e2a-503c-47ca-b069-96b11ae767c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:13.068470 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a36e2a-503c-47ca-b069-96b11ae767c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:13.069902 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a36e2a-503c-47ca-b069-96b11ae767c6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:13.079892 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 292/580] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:20:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:13.080980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a36e2a-503c-47ca-b069-96b11ae767c6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:13.097785 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:13.119683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:13.122424 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9873af2-28a1-4ab9-a010-0731f10d7308 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:13.161442 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfe704-a842-4d3b-a758-51c66537c975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:13.162709 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfe704-a842-4d3b-a758-51c66537c975 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:13.162998 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfe704-a842-4d3b-a758-51c66537c975 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:13.163269 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfe704-a842-4d3b-a758-51c66537c975 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:13.174513 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bfe704-a842-4d3b-a758-51c66537c975 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:13.175992 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bfe704-a842-4d3b-a758-51c66537c975 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:13.191943 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a36e2a-503c-47ca-b069-96b11ae767c6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:13.196170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-73bfe704-a842-4d3b-a758-51c66537c975 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:13.196170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:13.196170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:13.196170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:13.196170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:13.196170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:13.196170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:13.196170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:13.196170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:13.196170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:13.196170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:13.196170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:13.196170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:13.196170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:13.196170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:13.197087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:13.197087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:13.197087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:13.197087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:13.197087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:13.197087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:13.197087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:13.197087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:13.197087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:13.197087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:13.197087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:13.197087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:13.197087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:13.197087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:13.197087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:13.197087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:13.197087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:13.198637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:13.198637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:13.198637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:13.198637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:13.198637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 289/581] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:13.198637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfe704-a842-4d3b-a758-51c66537c975 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:13.231112 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4eb09-7be9-41e1-8166-a1710039f91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:13.231910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4eb09-7be9-41e1-8166-a1710039f91e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:13.232220 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4eb09-7be9-41e1-8166-a1710039f91e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:13.232489 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4eb09-7be9-41e1-8166-a1710039f91e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:13.242614 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4eb09-7be9-41e1-8166-a1710039f91e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:13.244059 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4eb09-7be9-41e1-8166-a1710039f91e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:13.262564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-45a4eb09-7be9-41e1-8166-a1710039f91e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:13.262564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:13.262564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:13.262564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:13.262564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:13.262564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:13.262564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:13.262564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:13.262564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:13.262564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:13.262564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:13.262564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:13.262564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:13.262564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:13.262564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:13.264311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:13.264311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:13.264311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:13.264311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:13.264311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:13.264311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:13.264311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:13.264311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:13.264311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:13.264311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:13.264311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:13.264311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:13.264311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:13.264311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:13.264311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:13.264311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:13.264311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:13.265575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:13.265575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:13.265575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:13.265575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:13.265575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 293/582] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:13.266266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4eb09-7be9-41e1-8166-a1710039f91e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:13.316894 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfe704-a842-4d3b-a758-51c66537c975 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:13.331750 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:13.351788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76743353-cd52-4244-9fd1-c3f35b2602d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:13.352449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76743353-cd52-4244-9fd1-c3f35b2602d2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:13.352731 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76743353-cd52-4244-9fd1-c3f35b2602d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:13.353198 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76743353-cd52-4244-9fd1-c3f35b2602d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:13.353929 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76743353-cd52-4244-9fd1-c3f35b2602d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:13.354385 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76743353-cd52-4244-9fd1-c3f35b2602d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:13.355798 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76743353-cd52-4244-9fd1-c3f35b2602d2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:13.367543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 290/583] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:13.367543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76743353-cd52-4244-9fd1-c3f35b2602d2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:13.375894 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4eb09-7be9-41e1-8166-a1710039f91e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:13.386825 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:13.411619 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbf96f-7227-4a36-b9b7-134d66abafb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:13.412414 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbf96f-7227-4a36-b9b7-134d66abafb1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:13.412731 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbf96f-7227-4a36-b9b7-134d66abafb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:13.413025 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbf96f-7227-4a36-b9b7-134d66abafb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:13.423559 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cbf96f-7227-4a36-b9b7-134d66abafb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:13.425322 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cbf96f-7227-4a36-b9b7-134d66abafb1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:13.443205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-b9cbf96f-7227-4a36-b9b7-134d66abafb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:13.443205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:13.443205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:13.443205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:13.443205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:13.443205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:13.443205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:13.443205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:13.443205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:13.443205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:13.443205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:13.443205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:13.443205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:13.443205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:13.443205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:13.444114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:13.444114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:13.444114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:13.444114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:13.444114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:13.444114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:13.444114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:13.444114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:13.444114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:13.444114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:13.444114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:13.444114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:13.444114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:13.444114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:13.444114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:13.444114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:13.444114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:13.444979 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:13.444979 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:13.444979 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:13.444979 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:13.444979 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 294/584] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:13.445706 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbf96f-7227-4a36-b9b7-134d66abafb1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:13.479750 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76743353-cd52-4244-9fd1-c3f35b2602d2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:13.517431 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f616f-0cb5-4abb-bd9c-33d21658d688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:13.518374 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f616f-0cb5-4abb-bd9c-33d21658d688 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:13.518654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f616f-0cb5-4abb-bd9c-33d21658d688 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:13.518929 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f616f-0cb5-4abb-bd9c-33d21658d688 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:13.519956 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f616f-0cb5-4abb-bd9c-33d21658d688 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:13.519956 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f616f-0cb5-4abb-bd9c-33d21658d688 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:13.521361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f616f-0cb5-4abb-bd9c-33d21658d688 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:13.538182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 291/585] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:13.538824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f616f-0cb5-4abb-bd9c-33d21658d688 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:13.552663 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbf96f-7227-4a36-b9b7-134d66abafb1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:13.570469 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:13.589887 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520f31b-6417-48d8-a30a-b3160afb9f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:13.590897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520f31b-6417-48d8-a30a-b3160afb9f91 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:13.591322 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520f31b-6417-48d8-a30a-b3160afb9f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:13.591605 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520f31b-6417-48d8-a30a-b3160afb9f91 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:13.592325 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520f31b-6417-48d8-a30a-b3160afb9f91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:13.592636 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520f31b-6417-48d8-a30a-b3160afb9f91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:13.594072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520f31b-6417-48d8-a30a-b3160afb9f91 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:13.606068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 295/586] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:13.606868 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520f31b-6417-48d8-a30a-b3160afb9f91 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:13.618849 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:13.649294 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f616f-0cb5-4abb-bd9c-33d21658d688 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:13.684391 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b5a33-09f9-492f-aa60-a495531e141c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:13.686041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b5a33-09f9-492f-aa60-a495531e141c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:13.686528 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b5a33-09f9-492f-aa60-a495531e141c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:13.687002 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b5a33-09f9-492f-aa60-a495531e141c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:13.694430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b5a33-09f9-492f-aa60-a495531e141c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:13.696136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b5a33-09f9-492f-aa60-a495531e141c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:13.711203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-7b1b5a33-09f9-492f-aa60-a495531e141c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:13.711203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:13.711203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:13.711203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:13.711203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:13.711203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:13.711203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:13.711203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:13.711203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:13.711203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:13.711203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:13.711203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:13.711203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:13.711203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:13.711203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:13.712065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:13.712065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:13.712065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:13.712065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:13.712065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:13.712065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:13.712065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:13.712065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:13.712065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:13.712065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:13.712065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:13.712065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:13.712065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:13.712065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:13.712065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:13.712065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:13.712065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:13.712942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:13.712942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:13.712942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:13.712942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:13.712942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:13.712942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:13.712942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:13.712942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:13.713508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520f31b-6417-48d8-a30a-b3160afb9f91 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:13.714959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 292/587] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:13.715871 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b5a33-09f9-492f-aa60-a495531e141c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:13.748471 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1bebd-55b1-4b24-afa0-d57e8175aea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:13.749166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1bebd-55b1-4b24-afa0-d57e8175aea3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:13.749453 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1bebd-55b1-4b24-afa0-d57e8175aea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:13.749730 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1bebd-55b1-4b24-afa0-d57e8175aea3 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:13.750445 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1bebd-55b1-4b24-afa0-d57e8175aea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:13.750754 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1bebd-55b1-4b24-afa0-d57e8175aea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:13.752116 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1bebd-55b1-4b24-afa0-d57e8175aea3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:13.763056 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 296/588] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:13.763890 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1bebd-55b1-4b24-afa0-d57e8175aea3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:13.800875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:13.827196 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b5a33-09f9-492f-aa60-a495531e141c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:13.859370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:13.864977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48130c19-4472-40a5-a869-c25688b70f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:13.865948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48130c19-4472-40a5-a869-c25688b70f45 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:13.866243 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48130c19-4472-40a5-a869-c25688b70f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:13.866498 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48130c19-4472-40a5-a869-c25688b70f45 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:13.868369 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1bebd-55b1-4b24-afa0-d57e8175aea3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:13.874640 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48130c19-4472-40a5-a869-c25688b70f45 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:13.876100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48130c19-4472-40a5-a869-c25688b70f45 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:13.892267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-48130c19-4472-40a5-a869-c25688b70f45 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 08:20:13.892267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:13.892267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:13.892267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:13.892267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:13.892267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:13.892267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:13.892267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:13.892267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:13.892267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:13.892267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:13.892267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:13.892267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:13.892267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:13.892267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:13.892267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:13.893472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:13.893472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:13.893472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:13.893472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:13.893472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:13.893472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:13.893472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:13.893472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:13.893472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:13.893472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:13.893472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:13.893472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:13.893472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:13.893472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:13.893472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:13.893472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:13.893472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:13.893472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:13.894485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:13.894485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:13.894485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:13.894485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:13.894485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:13.894485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:13.894485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:13.894485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:13.894485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 08:20:13.894485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:13.894485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 293/589] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:13.894485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48130c19-4472-40a5-a869-c25688b70f45 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:13.909884 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce08ca3-3a98-45df-bd41-fc4e80d1873c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:13.910684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce08ca3-3a98-45df-bd41-fc4e80d1873c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:13.910968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce08ca3-3a98-45df-bd41-fc4e80d1873c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:13.911244 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce08ca3-3a98-45df-bd41-fc4e80d1873c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:13.918300 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce08ca3-3a98-45df-bd41-fc4e80d1873c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:13.920288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce08ca3-3a98-45df-bd41-fc4e80d1873c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:13.934375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-7ce08ca3-3a98-45df-bd41-fc4e80d1873c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:13.934375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:13.934375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:13.934375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:13.934375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:13.934375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:13.934375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:13.934375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:13.934375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:13.934375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:13.934375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:13.934375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:13.934375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:13.934375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:13.934375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:13.935871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:13.935871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:13.935871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:13.935871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:13.935871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:13.935871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:13.935871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:13.935871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:13.935871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:13.935871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:13.935871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:13.935871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:13.935871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:13.935871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:13.935871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:13.935871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:13.935871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:13.941381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:13.941381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:13.941381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:13.941381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:13.941381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:13.941381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:13.941381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:13.941381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:13.941381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 297/590] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:13.941381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce08ca3-3a98-45df-bd41-fc4e80d1873c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:14.016031 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48130c19-4472-40a5-a869-c25688b70f45 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:14.038779 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:14.045405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce08ca3-3a98-45df-bd41-fc4e80d1873c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:14.056479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83b706-4766-4107-90ed-6122d4be5472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:14.057959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83b706-4766-4107-90ed-6122d4be5472 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:14.062399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83b706-4766-4107-90ed-6122d4be5472 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:14.062399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83b706-4766-4107-90ed-6122d4be5472 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:14.062399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83b706-4766-4107-90ed-6122d4be5472 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:14.062399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83b706-4766-4107-90ed-6122d4be5472 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:14.062399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83b706-4766-4107-90ed-6122d4be5472 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:14.087642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 294/591] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:14.087642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83b706-4766-4107-90ed-6122d4be5472 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:14.088024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45deabb6-0a0e-40ad-b387-33ce0be6378b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:14.088334 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45deabb6-0a0e-40ad-b387-33ce0be6378b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:14.088569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45deabb6-0a0e-40ad-b387-33ce0be6378b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:14.088892 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45deabb6-0a0e-40ad-b387-33ce0be6378b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:14.097441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45deabb6-0a0e-40ad-b387-33ce0be6378b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:14.098886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45deabb6-0a0e-40ad-b387-33ce0be6378b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:14.114248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-45deabb6-0a0e-40ad-b387-33ce0be6378b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 08:20:14.114248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:14.114248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:14.114248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:14.114248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:14.114248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:14.114248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:14.114248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:14.114248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:14.114248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:14.114248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:14.114248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:14.114248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:14.114248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:14.114248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:14.114248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:14.115299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:14.115299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:14.115299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:14.115299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:14.115299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:14.115299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:14.115299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:14.115299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:14.115299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:14.115299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:14.115299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:14.115299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:14.115299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:14.115299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:14.115299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:14.115299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:14.115299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:14.115299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:14.116279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:14.116279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:14.116279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:14.116279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:14.116279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:14.116279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:14.116279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:14.116279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:14.116279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 08:20:14.116279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:14.120254 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 298/592] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:14.121102 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45deabb6-0a0e-40ad-b387-33ce0be6378b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:14.139086 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:14.191802 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83b706-4766-4107-90ed-6122d4be5472 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:14.225452 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d29573-1db5-477f-ad65-5e0778e508ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:14.226368 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d29573-1db5-477f-ad65-5e0778e508ef None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:14.226827 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d29573-1db5-477f-ad65-5e0778e508ef None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:14.227263 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d29573-1db5-477f-ad65-5e0778e508ef None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:14.228257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d29573-1db5-477f-ad65-5e0778e508ef None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:14.228725 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d29573-1db5-477f-ad65-5e0778e508ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:14.229779 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45deabb6-0a0e-40ad-b387-33ce0be6378b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:14.230551 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d29573-1db5-477f-ad65-5e0778e508ef None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:14.250119 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 295/593] 10.222.0.17 () {56 vars in 1139 bytes} [Tue Jun 7 08:20:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:14.251659 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d29573-1db5-477f-ad65-5e0778e508ef None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:14.265168 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0d7df-6179-4ae9-b526-09a4b805a411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:14.265828 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0d7df-6179-4ae9-b526-09a4b805a411 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:14.266104 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0d7df-6179-4ae9-b526-09a4b805a411 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:14.266375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0d7df-6179-4ae9-b526-09a4b805a411 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:14.267157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0d7df-6179-4ae9-b526-09a4b805a411 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:14.267462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0d7df-6179-4ae9-b526-09a4b805a411 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:14.269016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0d7df-6179-4ae9-b526-09a4b805a411 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:14.281290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 299/594] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:14.282097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0d7df-6179-4ae9-b526-09a4b805a411 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:14.301122 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:14.366551 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d29573-1db5-477f-ad65-5e0778e508ef None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:14.375899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:14.387040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0d7df-6179-4ae9-b526-09a4b805a411 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:14.401351 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99f8d8-3db1-496b-9482-334b20694a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:14.402008 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99f8d8-3db1-496b-9482-334b20694a52 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:14.402293 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99f8d8-3db1-496b-9482-334b20694a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:14.402571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99f8d8-3db1-496b-9482-334b20694a52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:14.403405 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99f8d8-3db1-496b-9482-334b20694a52 None admin] RBAC: Adding query filter params (role.id=a04f2ab5f1234bd6b1dc4c18b4210a89, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=334dc47ae2e643ccb44d25c6b5906f14) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:14.403719 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99f8d8-3db1-496b-9482-334b20694a52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:14.405749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99f8d8-3db1-496b-9482-334b20694a52 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:14.412208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 296/595] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:14 2022] GET /identity/v3/role_assignments?user.id=334dc47ae2e643ccb44d25c6b5906f14&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:14.413582 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99f8d8-3db1-496b-9482-334b20694a52 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:14.427319 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede2eb6-3aa3-4dc4-b270-9fbbcf8fc5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:14.428305 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede2eb6-3aa3-4dc4-b270-9fbbcf8fc5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:14.428745 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede2eb6-3aa3-4dc4-b270-9fbbcf8fc5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:14.429211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede2eb6-3aa3-4dc4-b270-9fbbcf8fc5d1 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:14.430908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede2eb6-3aa3-4dc4-b270-9fbbcf8fc5d1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:14.431744 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede2eb6-3aa3-4dc4-b270-9fbbcf8fc5d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:14.433789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede2eb6-3aa3-4dc4-b270-9fbbcf8fc5d1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:14.463862 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 300/596] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Jun 7 08:20:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:14.463862 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede2eb6-3aa3-4dc4-b270-9fbbcf8fc5d1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:14.555102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99f8d8-3db1-496b-9482-334b20694a52 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:14.569663 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:14.602454 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd97d5-38dd-4079-ad25-157c09e906ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:14.603573 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd97d5-38dd-4079-ad25-157c09e906ff None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:14.603868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd97d5-38dd-4079-ad25-157c09e906ff None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:14.604149 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd97d5-38dd-4079-ad25-157c09e906ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:14.606349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede2eb6-3aa3-4dc4-b270-9fbbcf8fc5d1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:14.619916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd97d5-38dd-4079-ad25-157c09e906ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:14.621701 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd97d5-38dd-4079-ad25-157c09e906ff None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:14.648643 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6cc55-0675-4fa8-b941-3f97a47745fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:14.649635 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6cc55-0675-4fa8-b941-3f97a47745fc None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:14.649919 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6cc55-0675-4fa8-b941-3f97a47745fc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:14.650194 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6cc55-0675-4fa8-b941-3f97a47745fc None admin] PATH_INFO: `/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/c4d9b5e3826049f2800f1b9424977d75/roles/a04f2ab5f1234bd6b1dc4c18b4210a89` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:14.652400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-0abd97d5-38dd-4079-ad25-157c09e906ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:14.652400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:14.652400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:14.652400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:14.652400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:14.652400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:14.652400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:14.652400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:14.652400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:14.652400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:14.652400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:14.652400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:14.652400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:14.652400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:14.652400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:14.653679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:14.653679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:14.653679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:14.653679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:14.653679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:14.653679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:14.653679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:14.653679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:14.653679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:14.653679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:14.653679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:14.653679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:14.653679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:14.653679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:14.653679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:14.653679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:14.653679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:14.655216 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:14.655216 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:14.655216 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:14.655216 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:14.660540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 297/597] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:14.661778 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd97d5-38dd-4079-ad25-157c09e906ff None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:14.676369 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6cc55-0675-4fa8-b941-3f97a47745fc None admin] RBAC: Authorizing `identity:create_grant(project_id=afc8781e5f864d4a9374300ee9b5193b, user_id=c4d9b5e3826049f2800f1b9424977d75, role_id=a04f2ab5f1234bd6b1dc4c18b4210a89)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:14.678633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6cc55-0675-4fa8-b941-3f97a47745fc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:14.698646 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:14.742763 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 301/598] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:20:14 2022] PUT /identity/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/c4d9b5e3826049f2800f1b9424977d75/roles/a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:20:14.743759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6cc55-0675-4fa8-b941-3f97a47745fc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:14.782536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd97d5-38dd-4079-ad25-157c09e906ff None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:14.822699 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db80d7e-fbb0-403e-9434-aa2456104e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:14.824086 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db80d7e-fbb0-403e-9434-aa2456104e53 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:14.824583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db80d7e-fbb0-403e-9434-aa2456104e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:14.825186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db80d7e-fbb0-403e-9434-aa2456104e53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:14.838161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db80d7e-fbb0-403e-9434-aa2456104e53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:14.839947 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db80d7e-fbb0-403e-9434-aa2456104e53 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:14.860720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-9db80d7e-fbb0-403e-9434-aa2456104e53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:14.860720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:14.860720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:14.860720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:14.860720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:14.860720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:14.860720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:14.860720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:14.860720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:14.860720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:14.860720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:14.860720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:14.860720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:14.860720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:14.860720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:14.861706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:14.861706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:14.861706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:14.861706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:14.861706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:14.861706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:14.861706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:14.861706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:14.861706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:14.861706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:14.861706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:14.861706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:14.861706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:14.861706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:14.861706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:14.861706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:14.861706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:14.862621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:14.862621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:14.862621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:14.862621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:14.869311 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 298/599] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:14.871055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db80d7e-fbb0-403e-9434-aa2456104e53 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:14.878392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6cc55-0675-4fa8-b941-3f97a47745fc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:14.913941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:14.923365 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0c89c-27ee-48b3-a016-a8a898ea3690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:14.925274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0c89c-27ee-48b3-a016-a8a898ea3690 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:14.925565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0c89c-27ee-48b3-a016-a8a898ea3690 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:14.947285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0c89c-27ee-48b3-a016-a8a898ea3690 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:14.947285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e0c89c-27ee-48b3-a016-a8a898ea3690 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:14.947285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e0c89c-27ee-48b3-a016-a8a898ea3690 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:14.947285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e0c89c-27ee-48b3-a016-a8a898ea3690 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:14.947285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 302/600] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:14.952230 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0c89c-27ee-48b3-a016-a8a898ea3690 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:15.012285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:15.038856 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db80d7e-fbb0-403e-9434-aa2456104e53 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:15.086977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0c89c-27ee-48b3-a016-a8a898ea3690 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:15.087175 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad6a404-683d-41bf-b680-23865b80063c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:15.087668 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6a404-683d-41bf-b680-23865b80063c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:15.088011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6a404-683d-41bf-b680-23865b80063c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:15.088405 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6a404-683d-41bf-b680-23865b80063c None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:15.089178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad6a404-683d-41bf-b680-23865b80063c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:15.090358 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad6a404-683d-41bf-b680-23865b80063c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:15.090978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad6a404-683d-41bf-b680-23865b80063c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:15.104107 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 299/601] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:15.109040 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e1aea-ee0c-4b90-84d9-a3132124f888 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:15.109040 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e1aea-ee0c-4b90-84d9-a3132124f888 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:15.109040 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e1aea-ee0c-4b90-84d9-a3132124f888 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:15.109040 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 300/602] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:15.112269 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e1aea-ee0c-4b90-84d9-a3132124f888 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:15.132259 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20676f5-2993-447d-8284-96161d428782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:15.132973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20676f5-2993-447d-8284-96161d428782 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:15.133290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20676f5-2993-447d-8284-96161d428782 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:15.133559 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20676f5-2993-447d-8284-96161d428782 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:15.144307 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20676f5-2993-447d-8284-96161d428782 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:15.146645 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20676f5-2993-447d-8284-96161d428782 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:15.166577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a20676f5-2993-447d-8284-96161d428782 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:15.166577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:15.166577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:15.166577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:15.166577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:15.166577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:15.166577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:15.166577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:15.166577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:15.166577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:15.166577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:15.166577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:15.166577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:15.166577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:15.166577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:15.167507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:15.167507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:15.167507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:15.167507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:15.167507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:15.167507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:15.167507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:15.167507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:15.167507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:15.167507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:15.167507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:15.167507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:15.167507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:15.167507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:15.167507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:15.167507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:15.167507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:15.168384 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:15.168384 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:15.168384 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:15.168384 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:15.168384 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:15.168384 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:15.168384 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:15.168384 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:15.171014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 303/603] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:15.172033 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b083b-e7b1-4365-8536-eae452a695ea None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:15.172310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b083b-e7b1-4365-8536-eae452a695ea None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:15.172579 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b083b-e7b1-4365-8536-eae452a695ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:15.198437 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-cd0b083b-e7b1-4365-8536-eae452a695ea None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:15.199613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cd0b083b-e7b1-4365-8536-eae452a695ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:15.243067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:15.251458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e1aea-ee0c-4b90-84d9-a3132124f888 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:15.278509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 304/604] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:15.279739 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b083b-e7b1-4365-8536-eae452a695ea None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:15.311692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25900d4-ccc9-44da-87f7-7d65950b16f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:15.312542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25900d4-ccc9-44da-87f7-7d65950b16f3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:15.312835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25900d4-ccc9-44da-87f7-7d65950b16f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:15.313127 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25900d4-ccc9-44da-87f7-7d65950b16f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:15.320599 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25900d4-ccc9-44da-87f7-7d65950b16f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:15.322091 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25900d4-ccc9-44da-87f7-7d65950b16f3 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:15.324019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:15.336654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-d25900d4-ccc9-44da-87f7-7d65950b16f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:15.336654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:15.336654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:15.336654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:15.336654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:15.336654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:15.336654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:15.336654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:15.336654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:15.336654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:15.336654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:15.336654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:15.336654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:15.336654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:15.336654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:15.337610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:15.337610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:15.337610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:15.337610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:15.337610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:15.337610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:15.337610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:15.337610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:15.337610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:15.337610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:15.337610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:15.337610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:15.337610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:15.337610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:15.337610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:15.337610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:15.337610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:15.338469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:15.338469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:15.338469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:15.338469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:15.338469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:15.338469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:15.338469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:15.338469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:15.342880 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 301/605] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:15.343739 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25900d4-ccc9-44da-87f7-7d65950b16f3 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:15.416549 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b083b-e7b1-4365-8536-eae452a695ea None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:15.457677 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299dee7-1842-44da-a81d-fb96a6a81893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:15.458388 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299dee7-1842-44da-a81d-fb96a6a81893 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:15.458670 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299dee7-1842-44da-a81d-fb96a6a81893 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:15.458945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299dee7-1842-44da-a81d-fb96a6a81893 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:15.459731 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299dee7-1842-44da-a81d-fb96a6a81893 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:15.460039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299dee7-1842-44da-a81d-fb96a6a81893 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:15.461498 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299dee7-1842-44da-a81d-fb96a6a81893 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:15.464290 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25900d4-ccc9-44da-87f7-7d65950b16f3 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:15.474373 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 305/606] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:15.475193 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299dee7-1842-44da-a81d-fb96a6a81893 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:15.511740 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:15.515076 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072f71e-27a4-4a8d-a31b-ea896bf71cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:15.515875 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072f71e-27a4-4a8d-a31b-ea896bf71cae None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:15.516153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072f71e-27a4-4a8d-a31b-ea896bf71cae None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:15.516428 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072f71e-27a4-4a8d-a31b-ea896bf71cae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:15.533690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c072f71e-27a4-4a8d-a31b-ea896bf71cae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:15.545180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-c072f71e-27a4-4a8d-a31b-ea896bf71cae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:15.545180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:15.545180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:15.545180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:15.545180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:15.545180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:15.545180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:15.545180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:15.545180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:15.545180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:15.545180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:15.545180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:15.545180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:15.545180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:15.545180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:15.545180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:15.546223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:15.546223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:15.546223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:15.546223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:15.546223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:15.546223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:15.546223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:15.546223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:15.546223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:15.546223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:15.546223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:15.546223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:15.546223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:15.546223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:15.546223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:15.546223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:15.547073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:15.547073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:15.547073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:15.547073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:15.547073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:15.547073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:15.547073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:15.547073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:15.547073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:15.547073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:15.547073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:15.547073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:15.549201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 302/607] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:15.549995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072f71e-27a4-4a8d-a31b-ea896bf71cae None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:15.594190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299dee7-1842-44da-a81d-fb96a6a81893 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:15.623684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:15.639622 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ea650-5c52-46a8-bcc9-dfdeacbf1176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:15.640604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ea650-5c52-46a8-bcc9-dfdeacbf1176 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:15.641179 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ea650-5c52-46a8-bcc9-dfdeacbf1176 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:15.641612 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ea650-5c52-46a8-bcc9-dfdeacbf1176 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:15.643201 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ea650-5c52-46a8-bcc9-dfdeacbf1176 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:15.643714 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ea650-5c52-46a8-bcc9-dfdeacbf1176 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:15.645859 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ea650-5c52-46a8-bcc9-dfdeacbf1176 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:15.667659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 306/608] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:15.667659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ea650-5c52-46a8-bcc9-dfdeacbf1176 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:15.677682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072f71e-27a4-4a8d-a31b-ea896bf71cae None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:15.719178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03b786-eb4c-4ed7-bdaa-1373474654ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:15.720555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03b786-eb4c-4ed7-bdaa-1373474654ac None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:15.721094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03b786-eb4c-4ed7-bdaa-1373474654ac None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:15.721587 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03b786-eb4c-4ed7-bdaa-1373474654ac None admin] PATH_INFO: `/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/4a87f8bb06ee4a3ba7017422f4f1943e/roles/a04f2ab5f1234bd6b1dc4c18b4210a89` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:15.742722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03b786-eb4c-4ed7-bdaa-1373474654ac None admin] RBAC: Authorizing `identity:create_grant(project_id=afc8781e5f864d4a9374300ee9b5193b, user_id=4a87f8bb06ee4a3ba7017422f4f1943e, role_id=a04f2ab5f1234bd6b1dc4c18b4210a89)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:15.745360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03b786-eb4c-4ed7-bdaa-1373474654ac None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:15.782036 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:15.788712 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ea650-5c52-46a8-bcc9-dfdeacbf1176 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:15.822803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a054b5b-1396-42fa-beed-7aaea0c49236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:15.823526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a054b5b-1396-42fa-beed-7aaea0c49236 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:15.823856 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a054b5b-1396-42fa-beed-7aaea0c49236 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:15.824135 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a054b5b-1396-42fa-beed-7aaea0c49236 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:15.824875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a054b5b-1396-42fa-beed-7aaea0c49236 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:15.825207 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a054b5b-1396-42fa-beed-7aaea0c49236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:15.826605 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a054b5b-1396-42fa-beed-7aaea0c49236 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:15.840659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 307/609] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:20:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:15.840659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a054b5b-1396-42fa-beed-7aaea0c49236 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:15.843732 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 303/610] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:20:15 2022] PUT /identity/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/4a87f8bb06ee4a3ba7017422f4f1943e/roles/a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:20:15.844779 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03b786-eb4c-4ed7-bdaa-1373474654ac None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:16.034665 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a054b5b-1396-42fa-beed-7aaea0c49236 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:16.072009 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03b786-eb4c-4ed7-bdaa-1373474654ac None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:16.085024 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:16.111812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa8fe3-855a-4680-93f2-7c8f20da3a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:16.112711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa8fe3-855a-4680-93f2-7c8f20da3a0c None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:16.112989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa8fe3-855a-4680-93f2-7c8f20da3a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:16.113281 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa8fe3-855a-4680-93f2-7c8f20da3a0c None admin] PATH_INFO: `/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/85ea2962e7094f8c878d95680764c4e6/roles/a04f2ab5f1234bd6b1dc4c18b4210a89` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:16.129870 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668ca3c-5b85-40ba-b147-c3453ed58865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:16.131140 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa8fe3-855a-4680-93f2-7c8f20da3a0c None admin] RBAC: Authorizing `identity:create_grant(project_id=afc8781e5f864d4a9374300ee9b5193b, user_id=85ea2962e7094f8c878d95680764c4e6, role_id=a04f2ab5f1234bd6b1dc4c18b4210a89)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:16.131266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668ca3c-5b85-40ba-b147-c3453ed58865 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:16.131945 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668ca3c-5b85-40ba-b147-c3453ed58865 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:16.132422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668ca3c-5b85-40ba-b147-c3453ed58865 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:16.132592 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa8fe3-855a-4680-93f2-7c8f20da3a0c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:16.146167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668ca3c-5b85-40ba-b147-c3453ed58865 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:16.147639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668ca3c-5b85-40ba-b147-c3453ed58865 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:16.168098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-6668ca3c-5b85-40ba-b147-c3453ed58865 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:16.168098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:16.168098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:16.168098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:16.168098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:16.168098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:16.168098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:16.168098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:16.168098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:16.168098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:16.168098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:16.168098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:16.168098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:16.168098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:16.168098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:16.169055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:16.169055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:16.169055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:16.169055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:16.169055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:16.169055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:16.169055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:16.169055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:16.169055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:16.169055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:16.169055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:16.169055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:16.169055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:16.169055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:16.169055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:16.169055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:16.169055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:16.169989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:16.169989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:16.169989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:16.169989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:16.169989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 304/611] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:16.178214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668ca3c-5b85-40ba-b147-c3453ed58865 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:16.219844 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 308/612] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:20:15 2022] PUT /identity/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/85ea2962e7094f8c878d95680764c4e6/roles/a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:20:16.294425 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668ca3c-5b85-40ba-b147-c3453ed58865 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:16.331645 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ff410-c3b5-4da9-85fb-19c0d06b1fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:16.332307 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ff410-c3b5-4da9-85fb-19c0d06b1fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:16.332585 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ff410-c3b5-4da9-85fb-19c0d06b1fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:16.332857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ff410-c3b5-4da9-85fb-19c0d06b1fa7 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:16.333620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ff410-c3b5-4da9-85fb-19c0d06b1fa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:16.333925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ff410-c3b5-4da9-85fb-19c0d06b1fa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:16.335300 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ff410-c3b5-4da9-85fb-19c0d06b1fa7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:16.348116 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 305/613] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:16.353887 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa8fe3-855a-4680-93f2-7c8f20da3a0c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:16.363093 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:16.491997 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa8fe3-855a-4680-93f2-7c8f20da3a0c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:16.546179 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae3975-7f75-4e51-99a4-8478307cb2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:16.546909 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae3975-7f75-4e51-99a4-8478307cb2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:16.547190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae3975-7f75-4e51-99a4-8478307cb2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:16.547467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae3975-7f75-4e51-99a4-8478307cb2b6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:16.561954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae3975-7f75-4e51-99a4-8478307cb2b6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:16.563424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae3975-7f75-4e51-99a4-8478307cb2b6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:16.584354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-2fae3975-7f75-4e51-99a4-8478307cb2b6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:20:16.584354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:16.584354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:16.584354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:16.584354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:16.584354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:16.584354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:16.584354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:16.584354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:16.584354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:16.584354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:16.584354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:16.584354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:16.584354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:16.584354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:16.584354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:16.585345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:16.585345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:16.585345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:16.585345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:16.585345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:16.585345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:16.585345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:16.585345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:16.585345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:16.585345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:16.585345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:16.585345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:16.585345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:16.585345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:16.585345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:16.585345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:16.585345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:16.585345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:16.586277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:16.586277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:16.586277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:16.586277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:16.586277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:16.586277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:16.586277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:16.586277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:16.586277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:20:16.586277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:16.586277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 309/614] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Jun 7 08:20:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:16.592800 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ff410-c3b5-4da9-85fb-19c0d06b1fa7 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:16.627668 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:16.730803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ff410-c3b5-4da9-85fb-19c0d06b1fa7 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:16.774364 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29414e-553b-47b1-a81d-ec8fe0024a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:16.775397 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29414e-553b-47b1-a81d-ec8fe0024a5f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:16.775875 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29414e-553b-47b1-a81d-ec8fe0024a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:16.776369 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29414e-553b-47b1-a81d-ec8fe0024a5f None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:16.777423 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29414e-553b-47b1-a81d-ec8fe0024a5f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:16.777928 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29414e-553b-47b1-a81d-ec8fe0024a5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:16.779592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29414e-553b-47b1-a81d-ec8fe0024a5f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:16.803999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 306/615] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Jun 7 08:20:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:16.809757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae3975-7f75-4e51-99a4-8478307cb2b6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:16.891377 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836714ea-e56b-435b-b0b8-69163cea563c None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:16.891731 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836714ea-e56b-435b-b0b8-69163cea563c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:16.891926 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836714ea-e56b-435b-b0b8-69163cea563c None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:16.893448 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 307/616] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:16.905349 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca9da6-cdc7-440b-911a-8d6d7f219a34 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:16.905678 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca9da6-cdc7-440b-911a-8d6d7f219a34 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:16.905958 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca9da6-cdc7-440b-911a-8d6d7f219a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:16.928322 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:16.936760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-8dca9da6-cdc7-440b-911a-8d6d7f219a34 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:16.936760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8dca9da6-cdc7-440b-911a-8d6d7f219a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:16.989944 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae3975-7f75-4e51-99a4-8478307cb2b6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:17.057323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c4025-6c04-4ed6-aa02-414fbc22f521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:17.057937 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4025-6c04-4ed6-aa02-414fbc22f521 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:17.058161 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4025-6c04-4ed6-aa02-414fbc22f521 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:17.058430 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4025-6c04-4ed6-aa02-414fbc22f521 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:17.065784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 308/617] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:20:17.105102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca9da6-cdc7-440b-911a-8d6d7f219a34 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:17.123648 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:17.151294 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c4025-6c04-4ed6-aa02-414fbc22f521 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:17.152799 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c4025-6c04-4ed6-aa02-414fbc22f521 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:17.255806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:17.293075 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 310/618] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:20:17.299431 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c4025-6c04-4ed6-aa02-414fbc22f521 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:17.312604 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca9da6-cdc7-440b-911a-8d6d7f219a34 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:17.376644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4d765-c68d-492a-8a00-1ee070f86fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:17.377541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4d765-c68d-492a-8a00-1ee070f86fba None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:17.377826 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4d765-c68d-492a-8a00-1ee070f86fba None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:17.378099 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4d765-c68d-492a-8a00-1ee070f86fba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:17.395459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4d765-c68d-492a-8a00-1ee070f86fba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:17.403104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-9ab4d765-c68d-492a-8a00-1ee070f86fba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:17.403104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:17.403104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:17.403104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:17.403104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:17.403104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:17.403104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:17.403104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:17.403104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:17.403104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:17.403104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:17.403104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:17.403104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:17.403104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:17.403104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:17.403104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:17.404580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:17.404580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:17.404580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:17.404580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:17.404580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:17.404580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:17.404580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:17.404580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:17.404580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:17.404580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:17.404580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:17.404580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:17.404580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:17.404580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:17.404580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:17.404580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:17.405534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:17.405534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:17.405534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:17.405534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:17.405534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:17.405534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:17.405534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:17.405534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:17.405534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:17.405534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:17.405534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:17.405534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:17.409760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 309/619] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:17.411107 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97005417-7faa-4bff-b835-e5489284ac32 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:17.411394 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97005417-7faa-4bff-b835-e5489284ac32 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:17.411669 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97005417-7faa-4bff-b835-e5489284ac32 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:17.412379 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c4025-6c04-4ed6-aa02-414fbc22f521 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:17.413221 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 310/620] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:17.413926 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97005417-7faa-4bff-b835-e5489284ac32 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:17.451754 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e4c7b-2af0-4fef-84dd-b152c49342cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:17.452563 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e4c7b-2af0-4fef-84dd-b152c49342cb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:17.452951 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e4c7b-2af0-4fef-84dd-b152c49342cb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:17.453347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e4c7b-2af0-4fef-84dd-b152c49342cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:17.465177 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e4c7b-2af0-4fef-84dd-b152c49342cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:17.467259 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e4c7b-2af0-4fef-84dd-b152c49342cb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:17.486283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d41e4c7b-2af0-4fef-84dd-b152c49342cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:17.486283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:17.486283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:17.486283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:17.486283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:17.486283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:17.486283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:17.486283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:17.486283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:17.486283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:17.486283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:17.486283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:17.486283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:17.486283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:17.486283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:17.488321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:17.488321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:17.488321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:17.488321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:17.488321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:17.488321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:17.488321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:17.488321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:17.488321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:17.488321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:17.488321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:17.488321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:17.488321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:17.488321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:17.488321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:17.488321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:17.488321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:17.490229 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:17.490229 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:17.490229 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:17.490229 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:17.490739 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:17.497019 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 311/621] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:17.497019 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fb85f-64e4-4f66-8a68-4690a947cb2a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:17.497019 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fb85f-64e4-4f66-8a68-4690a947cb2a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:17.497019 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fb85f-64e4-4f66-8a68-4690a947cb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:17.511543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-741fb85f-64e4-4f66-8a68-4690a947cb2a None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:17.512749 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-741fb85f-64e4-4f66-8a68-4690a947cb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:17.523539 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97005417-7faa-4bff-b835-e5489284ac32 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:17.528007 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:17.558418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524819da-45eb-4722-854d-36e9d5585ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:17.559090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524819da-45eb-4722-854d-36e9d5585ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:17.559370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524819da-45eb-4722-854d-36e9d5585ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:17.559642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524819da-45eb-4722-854d-36e9d5585ec0 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:17.560428 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524819da-45eb-4722-854d-36e9d5585ec0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:17.560732 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524819da-45eb-4722-854d-36e9d5585ec0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:17.562167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524819da-45eb-4722-854d-36e9d5585ec0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:17.575439 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 311/622] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:20:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:17.576759 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524819da-45eb-4722-854d-36e9d5585ec0 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:17.587833 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 312/623] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:17.588686 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fb85f-64e4-4f66-8a68-4690a947cb2a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:17.694558 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fb85f-64e4-4f66-8a68-4690a947cb2a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:17.705952 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524819da-45eb-4722-854d-36e9d5585ec0 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:17.731219 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fdc86-41ec-4f78-948c-7efd4f38580d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:17.732065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fdc86-41ec-4f78-948c-7efd4f38580d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:17.732355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fdc86-41ec-4f78-948c-7efd4f38580d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:17.732634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fdc86-41ec-4f78-948c-7efd4f38580d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:17.735915 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:17.742187 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e2f4e-bbec-43ff-9b31-c5f4b4ce8672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:17.742886 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e2f4e-bbec-43ff-9b31-c5f4b4ce8672 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:17.743172 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e2f4e-bbec-43ff-9b31-c5f4b4ce8672 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:17.743445 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e2f4e-bbec-43ff-9b31-c5f4b4ce8672 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:17.744165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e2f4e-bbec-43ff-9b31-c5f4b4ce8672 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:17.744339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876fdc86-41ec-4f78-948c-7efd4f38580d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:17.744511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e2f4e-bbec-43ff-9b31-c5f4b4ce8672 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:17.745679 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876fdc86-41ec-4f78-948c-7efd4f38580d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:17.745943 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e2f4e-bbec-43ff-9b31-c5f4b4ce8672 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:17.755747 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:17.758645 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 312/624] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:17.759450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e2f4e-bbec-43ff-9b31-c5f4b4ce8672 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:17.771517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-876fdc86-41ec-4f78-948c-7efd4f38580d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:17.771517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:17.771517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:17.771517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:17.771517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:17.771517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:17.771517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:17.771517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:17.771517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:17.771517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:17.771517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:17.771517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:17.771517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:17.771517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:17.771517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:17.773506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:17.773506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:17.773506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:17.773506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:17.773506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:17.773506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:17.773506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:17.773506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:17.773506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:17.773506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:17.773506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:17.773506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:17.773506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:17.773506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:17.773506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:17.773506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:17.773506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:17.774793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:17.774793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:17.774793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:17.774793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:17.781015 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 313/625] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:17.781015 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fdc86-41ec-4f78-948c-7efd4f38580d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:17.900811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e2f4e-bbec-43ff-9b31-c5f4b4ce8672 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:17.910351 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fdc86-41ec-4f78-948c-7efd4f38580d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:17.946000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4827f-cf12-4a75-a9ba-4b8296b3efa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:17.947143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4827f-cf12-4a75-a9ba-4b8296b3efa8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:17.947866 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4827f-cf12-4a75-a9ba-4b8296b3efa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:17.948498 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4827f-cf12-4a75-a9ba-4b8296b3efa8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:17.953257 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b34ab-c0b6-49be-88a4-70a0f6b74165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:17.954048 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b34ab-c0b6-49be-88a4-70a0f6b74165 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:17.954468 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b34ab-c0b6-49be-88a4-70a0f6b74165 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:17.954751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b34ab-c0b6-49be-88a4-70a0f6b74165 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:17.961594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4827f-cf12-4a75-a9ba-4b8296b3efa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:17.962892 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b34ab-c0b6-49be-88a4-70a0f6b74165 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:17.964338 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b34ab-c0b6-49be-88a4-70a0f6b74165 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:17.970692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-39a4827f-cf12-4a75-a9ba-4b8296b3efa8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:17.970692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:17.970692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:17.970692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:17.970692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:17.970692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:17.970692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:17.970692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:17.970692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:17.970692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:17.970692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:17.970692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:17.970692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:17.970692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:17.970692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:17.970692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:17.972430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:17.972430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:17.972430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:17.972430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:17.972430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:17.972430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:17.972430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:17.972430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:17.972430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:17.972430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:17.972430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:17.972430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:17.972430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:17.972430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:17.972430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:17.972430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:17.973902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:17.973902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:17.973902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:17.973902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:17.973902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:17.973902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:17.973902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:17.973902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:17.973902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:17.973902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:17.973902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:17.973902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:17.973902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 313/626] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:17.973902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4827f-cf12-4a75-a9ba-4b8296b3efa8 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:17.985014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-4f5b34ab-c0b6-49be-88a4-70a0f6b74165 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:17.985014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:17.985014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:17.985014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:17.985014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:17.985014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:17.985014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:17.985014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:17.985014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:17.985014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:17.985014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:17.985014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:17.985014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:17.985014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:17.985014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:17.986768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:17.986768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:17.986768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:17.986768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:17.986768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:17.986768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:17.986768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:17.986768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:17.986768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:17.986768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:17.986768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:17.986768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:17.986768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:17.986768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:17.986768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:17.986768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:17.986768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:17.988768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:17.988768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:17.988768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:17.988768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:17.988768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:17.988768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:17.988768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:17.988768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:17.997062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 314/627] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:17.997062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b34ab-c0b6-49be-88a4-70a0f6b74165 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:18.032224 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:18.054323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:18.085477 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4827f-cf12-4a75-a9ba-4b8296b3efa8 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:18.100900 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b34ab-c0b6-49be-88a4-70a0f6b74165 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:18.120775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a6aaa-1f73-4991-acee-21ba0c48f368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:18.121525 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a6aaa-1f73-4991-acee-21ba0c48f368 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:18.121863 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a6aaa-1f73-4991-acee-21ba0c48f368 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:18.122139 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a6aaa-1f73-4991-acee-21ba0c48f368 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:18.123417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a6aaa-1f73-4991-acee-21ba0c48f368 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:18.123722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a6aaa-1f73-4991-acee-21ba0c48f368 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:18.125117 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a6aaa-1f73-4991-acee-21ba0c48f368 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:18.140135 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757a85a-75b6-46e3-b770-18448f5dd887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:18.140835 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757a85a-75b6-46e3-b770-18448f5dd887 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:18.141151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757a85a-75b6-46e3-b770-18448f5dd887 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:18.141322 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 314/628] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:18.141476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757a85a-75b6-46e3-b770-18448f5dd887 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:18.142161 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757a85a-75b6-46e3-b770-18448f5dd887 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:18.142335 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a6aaa-1f73-4991-acee-21ba0c48f368 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:18.142500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757a85a-75b6-46e3-b770-18448f5dd887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:18.144225 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757a85a-75b6-46e3-b770-18448f5dd887 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:18.153510 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 315/629] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:20:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:18.154302 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757a85a-75b6-46e3-b770-18448f5dd887 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:18.252631 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a6aaa-1f73-4991-acee-21ba0c48f368 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:18.264217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:18.272024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757a85a-75b6-46e3-b770-18448f5dd887 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:18.290740 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232ca5c-0303-42ae-a803-6249bac6a31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:18.291556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232ca5c-0303-42ae-a803-6249bac6a31b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:18.291837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232ca5c-0303-42ae-a803-6249bac6a31b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:18.292114 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232ca5c-0303-42ae-a803-6249bac6a31b None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:18.292928 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232ca5c-0303-42ae-a803-6249bac6a31b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:18.293284 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232ca5c-0303-42ae-a803-6249bac6a31b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:18.295313 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232ca5c-0303-42ae-a803-6249bac6a31b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:18.303921 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:18.314447 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 315/630] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:18.323651 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232ca5c-0303-42ae-a803-6249bac6a31b None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:18.323789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a56323-f046-4ce0-a6ea-b46242813d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:18.323789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a56323-f046-4ce0-a6ea-b46242813d4c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:18.323789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a56323-f046-4ce0-a6ea-b46242813d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:18.323789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a56323-f046-4ce0-a6ea-b46242813d4c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:18.328676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a56323-f046-4ce0-a6ea-b46242813d4c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:18.330391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a56323-f046-4ce0-a6ea-b46242813d4c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:18.347164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-17a56323-f046-4ce0-a6ea-b46242813d4c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:20:18.347164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:18.347164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:18.347164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:18.347164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:18.347164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:18.347164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:18.347164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:18.347164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:18.347164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:18.347164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:18.347164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:18.347164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:18.347164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:18.347164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:18.347164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:18.348355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:18.348355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:18.348355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:18.348355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:18.348355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:18.348355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:18.348355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:18.348355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:18.348355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:18.348355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:18.348355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:18.348355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:18.348355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:18.348355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:18.348355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:18.348355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:18.348355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:18.348355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:18.349524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:18.349524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:18.349524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:18.349524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:18.349524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:18.349524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:18.349524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:18.349524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:18.349524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:20:18.349524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:18.351867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 316/631] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:20:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:18.352902 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a56323-f046-4ce0-a6ea-b46242813d4c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:18.423614 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232ca5c-0303-42ae-a803-6249bac6a31b None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:18.460409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025100e2-fe7a-4fec-a3c6-97dab13d95f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:18.461082 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025100e2-fe7a-4fec-a3c6-97dab13d95f9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:18.461371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025100e2-fe7a-4fec-a3c6-97dab13d95f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:18.461650 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025100e2-fe7a-4fec-a3c6-97dab13d95f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:18.468527 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a56323-f046-4ce0-a6ea-b46242813d4c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:18.474902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025100e2-fe7a-4fec-a3c6-97dab13d95f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:18.476479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025100e2-fe7a-4fec-a3c6-97dab13d95f9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:18.495564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-025100e2-fe7a-4fec-a3c6-97dab13d95f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:18.495564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:18.495564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:18.495564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:18.495564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:18.495564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:18.495564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:18.495564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:18.495564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:18.495564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:18.495564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:18.495564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:18.495564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:18.495564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:18.495564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:18.496722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:18.496722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:18.496722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:18.496722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:18.496722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:18.496722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:18.496722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:18.496722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:18.496722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:18.496722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:18.496722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:18.496722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:18.496722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:18.496722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:18.496722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:18.496722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:18.496722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:18.497889 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:18.497889 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:18.497889 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:18.497889 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:18.501373 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 316/632] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:18.502215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025100e2-fe7a-4fec-a3c6-97dab13d95f9 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:18.510734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdafc31-c3b2-4e25-93e4-8cc3043a8555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:18.511710 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdafc31-c3b2-4e25-93e4-8cc3043a8555 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:18.512146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdafc31-c3b2-4e25-93e4-8cc3043a8555 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:18.512578 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdafc31-c3b2-4e25-93e4-8cc3043a8555 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:18.513795 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdafc31-c3b2-4e25-93e4-8cc3043a8555 None admin] RBAC: Adding query filter params (role.id=a04f2ab5f1234bd6b1dc4c18b4210a89, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=ee42568b059b49a198ed2c01d5efdafd) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:18.514280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdafc31-c3b2-4e25-93e4-8cc3043a8555 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:18.516038 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdafc31-c3b2-4e25-93e4-8cc3043a8555 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:18.523031 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 317/633] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:18 2022] GET /identity/v3/role_assignments?user.id=ee42568b059b49a198ed2c01d5efdafd&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:18.523968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdafc31-c3b2-4e25-93e4-8cc3043a8555 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:18.536889 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:18.578148 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:18.642726 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdafc31-c3b2-4e25-93e4-8cc3043a8555 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:18.655420 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025100e2-fe7a-4fec-a3c6-97dab13d95f9 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:18.684781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db03ee-6e27-4fa2-bb67-855cc620a449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:18.685815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db03ee-6e27-4fa2-bb67-855cc620a449 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:18.686283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db03ee-6e27-4fa2-bb67-855cc620a449 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:18.686726 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db03ee-6e27-4fa2-bb67-855cc620a449 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:18.687749 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db03ee-6e27-4fa2-bb67-855cc620a449 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:18.688293 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db03ee-6e27-4fa2-bb67-855cc620a449 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:18.690059 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db03ee-6e27-4fa2-bb67-855cc620a449 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:18.693256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2eb540-d452-4cc4-9316-a19eb836499d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:18.694177 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2eb540-d452-4cc4-9316-a19eb836499d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:18.694346 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2eb540-d452-4cc4-9316-a19eb836499d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:18.694690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2eb540-d452-4cc4-9316-a19eb836499d None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:18.695614 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2eb540-d452-4cc4-9316-a19eb836499d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:18.695991 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2eb540-d452-4cc4-9316-a19eb836499d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:18.697555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2eb540-d452-4cc4-9316-a19eb836499d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:18.706346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 318/634] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:18.707960 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da014c-b3eb-485d-8f35-264463b40143 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:18.708401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da014c-b3eb-485d-8f35-264463b40143 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:18.708937 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da014c-b3eb-485d-8f35-264463b40143 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:18.711500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 319/635] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:18.712689 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da014c-b3eb-485d-8f35-264463b40143 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:18.722875 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 317/636] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Jun 7 08:20:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:18.724369 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cb2ec-f2a5-4fe3-b36f-1e61d5bb40a6 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:18.724369 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cb2ec-f2a5-4fe3-b36f-1e61d5bb40a6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:18.724577 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cb2ec-f2a5-4fe3-b36f-1e61d5bb40a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:18.749055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-656cb2ec-f2a5-4fe3-b36f-1e61d5bb40a6 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:18.749055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-656cb2ec-f2a5-4fe3-b36f-1e61d5bb40a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:18.826335 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 318/637] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:18.827401 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656cb2ec-f2a5-4fe3-b36f-1e61d5bb40a6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:18.832794 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da014c-b3eb-485d-8f35-264463b40143 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:18.839271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:18.843329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:18.874253 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbb114-056f-4857-85a1-fa5ea13a2642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:18.875058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbb114-056f-4857-85a1-fa5ea13a2642 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:18.875340 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbb114-056f-4857-85a1-fa5ea13a2642 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:18.875614 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbb114-056f-4857-85a1-fa5ea13a2642 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:18.886258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adbb114-056f-4857-85a1-fa5ea13a2642 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:18.886258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adbb114-056f-4857-85a1-fa5ea13a2642 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:18.904500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5adbb114-056f-4857-85a1-fa5ea13a2642 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 08:20:18.904500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:18.904500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:18.904500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:18.904500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:18.904500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:18.904500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:18.904500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:18.904500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:18.904500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:18.904500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:18.904500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:18.904500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:18.904500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:18.904500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:18.904500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:18.905595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:18.905595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:18.905595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:18.905595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:18.905595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:18.905595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:18.905595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:18.905595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:18.905595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:18.905595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:18.905595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:18.905595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:18.905595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:18.905595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:18.905595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:18.905595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:18.905595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:18.905595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:18.906518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:18.906518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:18.906518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:18.906518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:18.906518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:18.906518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:18.906518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:18.906518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:18.906518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 08:20:18.906518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:18.908709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 320/638] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:18.910126 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47165888-6866-43b0-aa76-cc9655e456f9 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:18.910546 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47165888-6866-43b0-aa76-cc9655e456f9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:18.911060 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47165888-6866-43b0-aa76-cc9655e456f9 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:18.912319 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 321/639] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:18.913177 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47165888-6866-43b0-aa76-cc9655e456f9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:18.943245 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656cb2ec-f2a5-4fe3-b36f-1e61d5bb40a6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:18.983230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72073d-692c-4622-a798-f615d88c1e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:18.984440 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72073d-692c-4622-a798-f615d88c1e40 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:18.984725 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72073d-692c-4622-a798-f615d88c1e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:18.985208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72073d-692c-4622-a798-f615d88c1e40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:18.997147 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72073d-692c-4622-a798-f615d88c1e40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:18.999253 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72073d-692c-4622-a798-f615d88c1e40 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:19.017470 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47165888-6866-43b0-aa76-cc9655e456f9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:19.017720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-1f72073d-692c-4622-a798-f615d88c1e40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:19.017720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:19.017720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:19.017720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:19.017720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:19.017720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:19.017720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:19.017720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:19.017720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:19.017720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:19.017720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:19.017720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:19.017720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:19.017720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:19.017720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:19.018581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:19.018581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:19.018581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:19.018581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:19.018581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:19.018581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:19.018581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:19.018581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:19.018581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:19.018581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:19.018581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:19.018581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:19.018581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:19.018581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:19.018581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:19.018581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:19.018581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:19.019501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:19.019501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:19.019501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:19.019501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:19.019501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 319/640] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:19.019805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72073d-692c-4622-a798-f615d88c1e40 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:19.058120 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e193a16-c4d5-4ee3-87b6-4c05c372a4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:19.058921 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e193a16-c4d5-4ee3-87b6-4c05c372a4dc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:19.059203 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e193a16-c4d5-4ee3-87b6-4c05c372a4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:19.059476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e193a16-c4d5-4ee3-87b6-4c05c372a4dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:19.071135 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e193a16-c4d5-4ee3-87b6-4c05c372a4dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:19.079068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-2e193a16-c4d5-4ee3-87b6-4c05c372a4dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:19.079068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:19.079068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:19.079068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:19.079068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:19.079068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:19.079068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:19.079068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:19.079068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:19.079068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:19.079068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:19.079068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:19.079068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:19.079068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:19.079068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:19.079068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:19.080076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:19.080076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:19.080076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:19.080076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:19.080076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:19.080076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:19.080076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:19.080076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:19.080076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:19.080076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:19.080076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:19.080076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:19.080076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:19.080076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:19.080076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:19.080076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:19.080964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:19.080964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:19.080964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:19.080964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:19.080964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:19.080964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:19.080964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:19.080964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:19.080964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:19.080964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:19.080964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:19.080964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:19.080964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 322/641] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:19.082105 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5386765-f23a-4c7e-9497-cbb0de34ecad None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:19.082105 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5386765-f23a-4c7e-9497-cbb0de34ecad None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:19.082105 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5386765-f23a-4c7e-9497-cbb0de34ecad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:19.090978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:19.098238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f5386765-f23a-4c7e-9497-cbb0de34ecad None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:19.099488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f5386765-f23a-4c7e-9497-cbb0de34ecad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:19.120766 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:19.132932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72073d-692c-4622-a798-f615d88c1e40 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:19.167570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1e141-37c3-493c-9227-1b6c9dca1bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:19.167793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 323/642] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:19.173486 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1e141-37c3-493c-9227-1b6c9dca1bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:19.173642 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5386765-f23a-4c7e-9497-cbb0de34ecad None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:19.173733 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1e141-37c3-493c-9227-1b6c9dca1bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:19.173733 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1e141-37c3-493c-9227-1b6c9dca1bb1 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:19.173733 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1e141-37c3-493c-9227-1b6c9dca1bb1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:19.173733 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1e141-37c3-493c-9227-1b6c9dca1bb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:19.173733 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1e141-37c3-493c-9227-1b6c9dca1bb1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:19.202495 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 320/643] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Jun 7 08:20:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:19.203391 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1e141-37c3-493c-9227-1b6c9dca1bb1 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:19.287307 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5386765-f23a-4c7e-9497-cbb0de34ecad None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:19.319589 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1e141-37c3-493c-9227-1b6c9dca1bb1 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:19.323593 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba7b89-604a-4743-bf00-e9adcd571133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:19.324261 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba7b89-604a-4743-bf00-e9adcd571133 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:19.324545 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba7b89-604a-4743-bf00-e9adcd571133 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:19.324820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba7b89-604a-4743-bf00-e9adcd571133 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:19.325576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba7b89-604a-4743-bf00-e9adcd571133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:19.325913 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba7b89-604a-4743-bf00-e9adcd571133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:19.327694 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba7b89-604a-4743-bf00-e9adcd571133 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:19.339101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:19.346673 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 324/644] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:19.346673 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba7b89-604a-4743-bf00-e9adcd571133 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:19.353733 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb5223-1fb9-4546-abb5-57a8abeffa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:19.354427 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb5223-1fb9-4546-abb5-57a8abeffa10 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:19.354705 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb5223-1fb9-4546-abb5-57a8abeffa10 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:19.354983 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb5223-1fb9-4546-abb5-57a8abeffa10 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:19.356113 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb5223-1fb9-4546-abb5-57a8abeffa10 None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:19.356433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb5223-1fb9-4546-abb5-57a8abeffa10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:19.358831 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb5223-1fb9-4546-abb5-57a8abeffa10 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:19.365898 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:19.372626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 321/645] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:20:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:19.373812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb5223-1fb9-4546-abb5-57a8abeffa10 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:19.457819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba7b89-604a-4743-bf00-e9adcd571133 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:19.479034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb5223-1fb9-4546-abb5-57a8abeffa10 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:19.493821 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346d15d-6159-4b77-9e68-92c70bd4aa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:19.494614 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346d15d-6159-4b77-9e68-92c70bd4aa55 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:19.494896 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346d15d-6159-4b77-9e68-92c70bd4aa55 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:19.495166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346d15d-6159-4b77-9e68-92c70bd4aa55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:19.506756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346d15d-6159-4b77-9e68-92c70bd4aa55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:19.514391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9346d15d-6159-4b77-9e68-92c70bd4aa55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:19.514391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:19.514391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:19.514391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:19.514391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:19.514391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:19.514391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:19.514391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:19.514391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:19.514391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:19.514391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:19.514391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:19.514391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:19.514391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:19.514391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:19.514391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:19.515344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:19.515344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:19.515344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:19.515344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:19.515344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:19.515344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:19.515344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:19.515344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:19.515344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:19.515344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:19.515344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:19.515344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:19.515344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:19.515344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:19.515344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:19.515344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:19.521348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:19.521348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:19.521348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:19.521348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:19.521348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:19.521348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:19.521348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:19.521348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:19.521348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:19.521348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:19.521348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:19.521348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:19.522014 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a946607-9dd0-4635-9a70-944e92631900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:19.522014 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a946607-9dd0-4635-9a70-944e92631900 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:19.522014 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a946607-9dd0-4635-9a70-944e92631900 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:19.522214 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 325/646] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:19.522291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a946607-9dd0-4635-9a70-944e92631900 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:19.522362 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346d15d-6159-4b77-9e68-92c70bd4aa55 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:19.570616 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:19.590124 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a946607-9dd0-4635-9a70-944e92631900 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:19.592111 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a946607-9dd0-4635-9a70-944e92631900 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:19.598335 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:19.621049 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346d15d-6159-4b77-9e68-92c70bd4aa55 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:19.657623 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7647330-9ab0-4676-abff-7ea3830dde83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:19.658567 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7647330-9ab0-4676-abff-7ea3830dde83 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:19.658845 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7647330-9ab0-4676-abff-7ea3830dde83 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:19.659115 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7647330-9ab0-4676-abff-7ea3830dde83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:19.666280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7647330-9ab0-4676-abff-7ea3830dde83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:19.667824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7647330-9ab0-4676-abff-7ea3830dde83 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:19.683138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d7647330-9ab0-4676-abff-7ea3830dde83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:19.683138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:19.683138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:19.683138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:19.683138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:19.683138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:19.683138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:19.683138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:19.683138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:19.683138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:19.683138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:19.683138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:19.683138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:19.683138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:19.683138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:19.684022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:19.684022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:19.684022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:19.684022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:19.684022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:19.684022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:19.684022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:19.684022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:19.684022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:19.684022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:19.684022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:19.684022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:19.684022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:19.684022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:19.684022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:19.684022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:19.684022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:19.684897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:19.684897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:19.684897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:19.684897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:19.684897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:19.684897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:19.684897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:19.684897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:19.684897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 326/647] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:19.685941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7647330-9ab0-4676-abff-7ea3830dde83 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:19.701708 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 322/648] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:20:19.701708 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a946607-9dd0-4635-9a70-944e92631900 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:19.783210 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:19.794067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7647330-9ab0-4676-abff-7ea3830dde83 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:19.803151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a946607-9dd0-4635-9a70-944e92631900 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:19.830234 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da755fa3-e5fe-41cd-89ff-03e07be07ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:19.831134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da755fa3-e5fe-41cd-89ff-03e07be07ebd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:19.831579 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da755fa3-e5fe-41cd-89ff-03e07be07ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:19.831852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da755fa3-e5fe-41cd-89ff-03e07be07ebd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:19.834690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:19.840481 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060e3c3-f6b6-4d8b-a013-ce18065c6dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:19.841180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060e3c3-f6b6-4d8b-a013-ce18065c6dac None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:19.842016 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060e3c3-f6b6-4d8b-a013-ce18065c6dac None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:19.842293 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060e3c3-f6b6-4d8b-a013-ce18065c6dac None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:19.843251 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1060e3c3-f6b6-4d8b-a013-ce18065c6dac None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:19.843447 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da755fa3-e5fe-41cd-89ff-03e07be07ebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:19.843548 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1060e3c3-f6b6-4d8b-a013-ce18065c6dac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:19.844888 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da755fa3-e5fe-41cd-89ff-03e07be07ebd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:19.844983 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1060e3c3-f6b6-4d8b-a013-ce18065c6dac None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:19.855913 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 323/649] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:20:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:19.857044 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060e3c3-f6b6-4d8b-a013-ce18065c6dac None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:19.862412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-da755fa3-e5fe-41cd-89ff-03e07be07ebd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:19.862412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:19.862412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:19.862412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:19.862412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:19.862412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:19.862412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:19.862412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:19.862412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:19.862412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:19.862412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:19.862412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:19.862412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:19.862412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:19.862412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:19.863380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:19.863380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:19.863380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:19.863380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:19.863380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:19.863380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:19.863380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:19.863380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:19.863380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:19.863380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:19.863380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:19.863380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:19.863380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:19.863380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:19.863380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:19.863380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:19.863380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:19.864255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:19.864255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:19.864255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:19.864255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:19.868085 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 327/650] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:19.868949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da755fa3-e5fe-41cd-89ff-03e07be07ebd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:19.963545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060e3c3-f6b6-4d8b-a013-ce18065c6dac None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:19.980270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da755fa3-e5fe-41cd-89ff-03e07be07ebd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:19.999420 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71b66f-e869-423a-b1dc-7346c4ea3346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:20.000079 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71b66f-e869-423a-b1dc-7346c4ea3346 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:20.000364 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71b66f-e869-423a-b1dc-7346c4ea3346 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:20.000642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71b66f-e869-423a-b1dc-7346c4ea3346 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:20.001872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71b66f-e869-423a-b1dc-7346c4ea3346 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:20.002337 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71b66f-e869-423a-b1dc-7346c4ea3346 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:20.004061 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71b66f-e869-423a-b1dc-7346c4ea3346 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:20.015194 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404f777-aa37-470a-96ed-4d0e536fd63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:20.015986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404f777-aa37-470a-96ed-4d0e536fd63d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:20.016266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404f777-aa37-470a-96ed-4d0e536fd63d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:20.016541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404f777-aa37-470a-96ed-4d0e536fd63d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:20.020569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 324/651] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:20.021648 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71b66f-e869-423a-b1dc-7346c4ea3346 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:20.026936 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:20.027083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404f777-aa37-470a-96ed-4d0e536fd63d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:20.028526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404f777-aa37-470a-96ed-4d0e536fd63d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:20.047555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-6404f777-aa37-470a-96ed-4d0e536fd63d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:20.047555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:20.047555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:20.047555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:20.047555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:20.047555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:20.047555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:20.047555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:20.047555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:20.047555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:20.047555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:20.047555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:20.047555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:20.047555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:20.047555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:20.048474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:20.048474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:20.048474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:20.048474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:20.048474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:20.048474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:20.048474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:20.048474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:20.048474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:20.048474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:20.048474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:20.048474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:20.048474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:20.048474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:20.048474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:20.048474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:20.048474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:20.049955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:20.049955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:20.049955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:20.049955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:20.049955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 328/652] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:20.050234 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404f777-aa37-470a-96ed-4d0e536fd63d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:20.094902 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:20.132930 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71b66f-e869-423a-b1dc-7346c4ea3346 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:20.152902 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404f777-aa37-470a-96ed-4d0e536fd63d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:20.167607 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5d67a-7b2c-4195-be87-a714393391e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:20.168396 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5d67a-7b2c-4195-be87-a714393391e1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:20.168676 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5d67a-7b2c-4195-be87-a714393391e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:20.168952 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5d67a-7b2c-4195-be87-a714393391e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:20.180178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5d67a-7b2c-4195-be87-a714393391e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:20.181674 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5d67a-7b2c-4195-be87-a714393391e1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:20.193369 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a74735-00b4-4b37-b179-cb69bf9f123e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:20.194038 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a74735-00b4-4b37-b179-cb69bf9f123e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:20.194323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a74735-00b4-4b37-b179-cb69bf9f123e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:20.194604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a74735-00b4-4b37-b179-cb69bf9f123e None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:20.195328 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a74735-00b4-4b37-b179-cb69bf9f123e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:20.195637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a74735-00b4-4b37-b179-cb69bf9f123e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:20.197034 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a74735-00b4-4b37-b179-cb69bf9f123e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:20.199285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-3fc5d67a-7b2c-4195-be87-a714393391e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:20.199285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:20.199285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:20.199285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:20.199285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:20.199285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:20.199285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:20.199285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:20.199285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:20.199285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:20.199285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:20.199285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:20.199285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:20.199285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:20.199285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:20.200171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:20.200171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:20.200171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:20.200171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:20.200171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:20.200171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:20.200171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:20.200171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:20.200171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:20.200171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:20.200171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:20.200171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:20.200171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:20.200171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:20.200171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:20.200171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:20.200171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:20.201104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:20.201104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:20.201104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:20.201104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:20.205075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 325/653] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:20.205697 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5d67a-7b2c-4195-be87-a714393391e1 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:20.215209 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 329/654] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:20.215209 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a74735-00b4-4b37-b179-cb69bf9f123e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:20.264952 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:20.312007 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5d67a-7b2c-4195-be87-a714393391e1 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:20.321347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:20.324228 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a74735-00b4-4b37-b179-cb69bf9f123e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:20.350144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09891b82-5df9-446f-b39b-319a7ba0b296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:20.351028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09891b82-5df9-446f-b39b-319a7ba0b296 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:20.351310 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09891b82-5df9-446f-b39b-319a7ba0b296 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:20.351585 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09891b82-5df9-446f-b39b-319a7ba0b296 None admin] PATH_INFO: `/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/334dc47ae2e643ccb44d25c6b5906f14/roles/a04f2ab5f1234bd6b1dc4c18b4210a89` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:20.359864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141287a-337d-4605-9a20-868ed35ccf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:20.360524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141287a-337d-4605-9a20-868ed35ccf6b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:20.360806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141287a-337d-4605-9a20-868ed35ccf6b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:20.361116 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141287a-337d-4605-9a20-868ed35ccf6b None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:20.361830 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141287a-337d-4605-9a20-868ed35ccf6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:20.362132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141287a-337d-4605-9a20-868ed35ccf6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:20.363502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141287a-337d-4605-9a20-868ed35ccf6b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:20.368643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09891b82-5df9-446f-b39b-319a7ba0b296 None admin] RBAC: Authorizing `identity:create_grant(project_id=afc8781e5f864d4a9374300ee9b5193b, user_id=334dc47ae2e643ccb44d25c6b5906f14, role_id=a04f2ab5f1234bd6b1dc4c18b4210a89)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:20.370932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09891b82-5df9-446f-b39b-319a7ba0b296 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:20.374701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 330/655] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:20.375501 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141287a-337d-4605-9a20-868ed35ccf6b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:20.453281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 326/656] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:20:20 2022] PUT /identity/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/334dc47ae2e643ccb44d25c6b5906f14/roles/a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:20:20.454746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09891b82-5df9-446f-b39b-319a7ba0b296 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:20.483035 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141287a-337d-4605-9a20-868ed35ccf6b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:20.533931 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef700ab-6d98-4395-8427-5fd0910ec43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:20.534764 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef700ab-6d98-4395-8427-5fd0910ec43f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:20.535690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef700ab-6d98-4395-8427-5fd0910ec43f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:20.535975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef700ab-6d98-4395-8427-5fd0910ec43f None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:20.537041 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef700ab-6d98-4395-8427-5fd0910ec43f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:20.537363 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef700ab-6d98-4395-8427-5fd0910ec43f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:20.539564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef700ab-6d98-4395-8427-5fd0910ec43f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:20.557914 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 331/657] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:20.558796 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef700ab-6d98-4395-8427-5fd0910ec43f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:20.587420 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:20.600063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09891b82-5df9-446f-b39b-319a7ba0b296 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:20.609121 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:20.642527 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d618d-3536-4ec3-b6f7-81b34cb5b476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:20.643757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d618d-3536-4ec3-b6f7-81b34cb5b476 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:20.644264 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d618d-3536-4ec3-b6f7-81b34cb5b476 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:20.644725 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d618d-3536-4ec3-b6f7-81b34cb5b476 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:20.654993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d618d-3536-4ec3-b6f7-81b34cb5b476 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:20.657030 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d618d-3536-4ec3-b6f7-81b34cb5b476 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:20.674653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-6d8d618d-3536-4ec3-b6f7-81b34cb5b476 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 08:20:20.674653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:20.674653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:20.674653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:20.674653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:20.674653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:20.674653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:20.674653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:20.674653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:20.674653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:20.674653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:20.674653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:20.674653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:20.674653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:20.674653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:20.674653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:20.675685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:20.675685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:20.675685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:20.675685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:20.675685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:20.675685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:20.675685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:20.675685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:20.675685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:20.675685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:20.675685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:20.675685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:20.675685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:20.675685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:20.675685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:20.675685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:20.675685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:20.675685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:20.676626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:20.676626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:20.676626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:20.676626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:20.676626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:20.676626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:20.676626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:20.676626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:20.676626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 08:20:20.676626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:20.677209 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef700ab-6d98-4395-8427-5fd0910ec43f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:20.679243 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 327/658] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:20.680263 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d618d-3536-4ec3-b6f7-81b34cb5b476 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:20.721702 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf181e-4c6c-4c22-b49c-ec5cc0cdf0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:20.722507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf181e-4c6c-4c22-b49c-ec5cc0cdf0b5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:20.722784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf181e-4c6c-4c22-b49c-ec5cc0cdf0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:20.723054 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf181e-4c6c-4c22-b49c-ec5cc0cdf0b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:20.730346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf181e-4c6c-4c22-b49c-ec5cc0cdf0b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:20.731935 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf181e-4c6c-4c22-b49c-ec5cc0cdf0b5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:20.746279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8fdf181e-4c6c-4c22-b49c-ec5cc0cdf0b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:20.746279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:20.746279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:20.746279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:20.746279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:20.746279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:20.746279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:20.746279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:20.746279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:20.746279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:20.746279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:20.746279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:20.746279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:20.746279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:20.746279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:20.747157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:20.747157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:20.747157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:20.747157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:20.747157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:20.747157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:20.747157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:20.747157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:20.747157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:20.747157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:20.747157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:20.747157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:20.747157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:20.747157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:20.747157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:20.747157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:20.747157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:20.748591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:20.748591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:20.748591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:20.748591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:20.748591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:20.748591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:20.748591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:20.748591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:20.748591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 332/659] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:20.749101 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf181e-4c6c-4c22-b49c-ec5cc0cdf0b5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:20.794096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d618d-3536-4ec3-b6f7-81b34cb5b476 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:20.838601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f24014-5702-46de-a9ac-91a20b54a6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:20.839594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f24014-5702-46de-a9ac-91a20b54a6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:20.839876 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f24014-5702-46de-a9ac-91a20b54a6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:20.840151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f24014-5702-46de-a9ac-91a20b54a6a3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:20.840888 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:20.849280 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f24014-5702-46de-a9ac-91a20b54a6a3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:20.853329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f24014-5702-46de-a9ac-91a20b54a6a3 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:20.861290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf181e-4c6c-4c22-b49c-ec5cc0cdf0b5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:20.869138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-15f24014-5702-46de-a9ac-91a20b54a6a3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 08:20:20.869138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:20.869138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:20.869138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:20.869138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:20.869138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:20.869138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:20.869138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:20.869138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:20.869138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:20.869138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:20.869138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:20.869138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:20.869138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:20.869138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:20.869138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:20.870292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:20.870292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:20.870292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:20.870292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:20.870292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:20.870292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:20.870292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:20.870292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:20.870292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:20.870292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:20.870292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:20.870292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:20.870292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:20.870292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:20.870292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:20.870292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:20.870292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:20.870292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:20.876540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:20.876540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:20.876540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:20.876540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:20.876540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:20.876540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:20.876540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:20.876540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:20.876540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 08:20:20.876540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:20.876540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 328/660] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:20.876540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f24014-5702-46de-a9ac-91a20b54a6a3 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:20.906069 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb5a9f-0175-4e14-9409-d2c5b2048c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:20.907194 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb5a9f-0175-4e14-9409-d2c5b2048c20 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:20.907705 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb5a9f-0175-4e14-9409-d2c5b2048c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:20.908184 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb5a9f-0175-4e14-9409-d2c5b2048c20 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:20.908972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:20.910258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb5a9f-0175-4e14-9409-d2c5b2048c20 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:20.910832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb5a9f-0175-4e14-9409-d2c5b2048c20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:20.912625 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb5a9f-0175-4e14-9409-d2c5b2048c20 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:20.932259 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 333/661] 10.222.0.17 () {56 vars in 1139 bytes} [Tue Jun 7 08:20:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:20.933737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb5a9f-0175-4e14-9409-d2c5b2048c20 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:20.995237 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f24014-5702-46de-a9ac-91a20b54a6a3 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:21.033255 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296b5e6-cd2b-4940-ab26-e2bda99c0baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:21.034283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296b5e6-cd2b-4940-ab26-e2bda99c0baf None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:21.034791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296b5e6-cd2b-4940-ab26-e2bda99c0baf None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:21.035268 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296b5e6-cd2b-4940-ab26-e2bda99c0baf None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:21.036237 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296b5e6-cd2b-4940-ab26-e2bda99c0baf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:21.036753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296b5e6-cd2b-4940-ab26-e2bda99c0baf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:21.038333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296b5e6-cd2b-4940-ab26-e2bda99c0baf None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:21.039024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb5a9f-0175-4e14-9409-d2c5b2048c20 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:21.055537 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 329/662] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:21.057149 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14ded1-59d4-43a5-901a-fcad18c77983 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:21.057542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14ded1-59d4-43a5-901a-fcad18c77983 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:21.057921 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14ded1-59d4-43a5-901a-fcad18c77983 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:21.059569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 330/663] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:21.060901 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14ded1-59d4-43a5-901a-fcad18c77983 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:21.079888 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665976f-bef5-4bf5-b161-ddc42ad0f4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:21.080558 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665976f-bef5-4bf5-b161-ddc42ad0f4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:21.080853 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665976f-bef5-4bf5-b161-ddc42ad0f4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:21.081144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665976f-bef5-4bf5-b161-ddc42ad0f4f8 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:21.081975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d665976f-bef5-4bf5-b161-ddc42ad0f4f8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:21.082278 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d665976f-bef5-4bf5-b161-ddc42ad0f4f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:21.083695 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d665976f-bef5-4bf5-b161-ddc42ad0f4f8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:21.092167 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:21.109066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 334/664] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Jun 7 08:20:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:21.109066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665976f-bef5-4bf5-b161-ddc42ad0f4f8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:21.151307 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:21.166975 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14ded1-59d4-43a5-901a-fcad18c77983 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:21.205309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25963127-2af2-4ff1-9a75-1534e3da8700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:21.205956 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25963127-2af2-4ff1-9a75-1534e3da8700 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:21.206240 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25963127-2af2-4ff1-9a75-1534e3da8700 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:21.206519 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25963127-2af2-4ff1-9a75-1534e3da8700 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:21.219202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665976f-bef5-4bf5-b161-ddc42ad0f4f8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:21.254537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdbd13b-911f-42b6-a497-7cc3b115442c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:21.255479 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbd13b-911f-42b6-a497-7cc3b115442c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:21.255911 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbd13b-911f-42b6-a497-7cc3b115442c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:21.256389 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbd13b-911f-42b6-a497-7cc3b115442c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:21.257394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdbd13b-911f-42b6-a497-7cc3b115442c None admin] RBAC: Adding query filter params (role.id=a04f2ab5f1234bd6b1dc4c18b4210a89, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=c4d9b5e3826049f2800f1b9424977d75) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:21.257885 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdbd13b-911f-42b6-a497-7cc3b115442c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:21.259584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdbd13b-911f-42b6-a497-7cc3b115442c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:21.273214 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 335/665] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:21 2022] GET /identity/v3/role_assignments?user.id=c4d9b5e3826049f2800f1b9424977d75&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:21.274727 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9bbd2-102e-4b11-8aed-a2bada35324a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:21.275011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9bbd2-102e-4b11-8aed-a2bada35324a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:21.275285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9bbd2-102e-4b11-8aed-a2bada35324a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:21.284256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25963127-2af2-4ff1-9a75-1534e3da8700 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:21.286841 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25963127-2af2-4ff1-9a75-1534e3da8700 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:21.295892 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3eb9bbd2-102e-4b11-8aed-a2bada35324a None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:21.297815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3eb9bbd2-102e-4b11-8aed-a2bada35324a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:21.381181 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:21.388496 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:21.401041 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 336/666] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:21.401041 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9bbd2-102e-4b11-8aed-a2bada35324a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:21.421861 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 331/667] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:20:21.428228 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25963127-2af2-4ff1-9a75-1534e3da8700 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:21.525631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9bbd2-102e-4b11-8aed-a2bada35324a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:21.557763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25963127-2af2-4ff1-9a75-1534e3da8700 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:21.567935 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fedefe-2401-4059-8270-e9aa966303fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:21.568967 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fedefe-2401-4059-8270-e9aa966303fd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:21.569484 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fedefe-2401-4059-8270-e9aa966303fd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:21.569975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fedefe-2401-4059-8270-e9aa966303fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:21.600570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317d9ae-14cf-4b89-9b88-3ba6550548f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:21.601397 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317d9ae-14cf-4b89-9b88-3ba6550548f0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:21.601682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317d9ae-14cf-4b89-9b88-3ba6550548f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:21.602011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317d9ae-14cf-4b89-9b88-3ba6550548f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:21.619841 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8317d9ae-14cf-4b89-9b88-3ba6550548f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:21.621475 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8317d9ae-14cf-4b89-9b88-3ba6550548f0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:21.636678 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:21.641028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-8317d9ae-14cf-4b89-9b88-3ba6550548f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:21.641028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:21.641028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:21.641028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:21.641028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:21.641028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:21.641028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:21.641028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:21.641028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:21.641028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:21.641028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:21.641028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:21.641028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:21.641028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:21.641028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:21.641896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:21.641896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:21.641896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:21.641896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:21.641896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:21.641896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:21.641896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:21.641896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:21.641896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:21.641896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:21.641896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:21.641896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:21.641896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:21.641896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:21.641896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:21.641896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:21.641896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:21.642767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:21.642767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:21.642767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:21.642767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:21.646591 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 332/668] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:21.647443 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317d9ae-14cf-4b89-9b88-3ba6550548f0 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:21.655436 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fedefe-2401-4059-8270-e9aa966303fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:21.655436 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fedefe-2401-4059-8270-e9aa966303fd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:21.693185 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:21.758451 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317d9ae-14cf-4b89-9b88-3ba6550548f0 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:21.760143 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 337/669] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:20:21.761286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fedefe-2401-4059-8270-e9aa966303fd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:21.802626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f350f-2804-447e-9238-689a31960af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:21.803430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f350f-2804-447e-9238-689a31960af5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:21.803713 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f350f-2804-447e-9238-689a31960af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:21.803989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f350f-2804-447e-9238-689a31960af5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:21.817740 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f350f-2804-447e-9238-689a31960af5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:21.827862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-424f350f-2804-447e-9238-689a31960af5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:21.827862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:21.827862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:21.827862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:21.827862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:21.827862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:21.827862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:21.827862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:21.827862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:21.827862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:21.827862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:21.827862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:21.827862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:21.827862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:21.827862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:21.827862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:21.828839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:21.828839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:21.828839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:21.828839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:21.828839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:21.828839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:21.828839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:21.828839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:21.828839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:21.828839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:21.828839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:21.828839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:21.828839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:21.828839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:21.828839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:21.828839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:21.830621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:21.830621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:21.830621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:21.830621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:21.830621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:21.830621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:21.830621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:21.830621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:21.830621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:21.830621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:21.830621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:21.830621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:21.830621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 333/670] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:20:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:21.830621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f350f-2804-447e-9238-689a31960af5 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:21.874044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:21.885637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fedefe-2401-4059-8270-e9aa966303fd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:21.927337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565ebd6-041c-4870-9612-63511ddbdb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:21.928005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565ebd6-041c-4870-9612-63511ddbdb42 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:21.928283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565ebd6-041c-4870-9612-63511ddbdb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:21.928556 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565ebd6-041c-4870-9612-63511ddbdb42 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:21.929312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565ebd6-041c-4870-9612-63511ddbdb42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:21.929618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565ebd6-041c-4870-9612-63511ddbdb42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:21.930977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565ebd6-041c-4870-9612-63511ddbdb42 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:21.946520 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 338/671] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:21.946520 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565ebd6-041c-4870-9612-63511ddbdb42 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:21.974015 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:21.977888 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f350f-2804-447e-9238-689a31960af5 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:22.012137 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256f19f-3ba8-4871-8030-d88e126cc38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:22.012928 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256f19f-3ba8-4871-8030-d88e126cc38d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:22.013758 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256f19f-3ba8-4871-8030-d88e126cc38d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:22.014033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256f19f-3ba8-4871-8030-d88e126cc38d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:22.024615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256f19f-3ba8-4871-8030-d88e126cc38d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:22.026205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256f19f-3ba8-4871-8030-d88e126cc38d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:22.046510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-5256f19f-3ba8-4871-8030-d88e126cc38d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:22.046510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:22.046510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:22.046510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:22.046510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:22.046510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:22.046510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:22.046510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:22.046510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:22.046510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:22.046510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:22.046510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:22.046510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:22.046510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:22.046510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:22.047419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:22.047419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:22.047419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:22.047419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:22.047419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:22.047419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:22.047419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:22.047419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:22.047419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:22.047419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:22.047419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:22.047419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:22.047419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:22.047419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:22.047419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:22.047419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:22.047419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:22.048318 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:22.048318 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:22.048318 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:22.048318 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:22.048318 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 334/672] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:22.049193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256f19f-3ba8-4871-8030-d88e126cc38d None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:22.071281 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565ebd6-041c-4870-9612-63511ddbdb42 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:22.116001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae15107-a44e-4da3-bda8-eb35d1e77692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:22.117123 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae15107-a44e-4da3-bda8-eb35d1e77692 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:22.117551 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae15107-a44e-4da3-bda8-eb35d1e77692 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:22.118443 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae15107-a44e-4da3-bda8-eb35d1e77692 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:22.119619 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae15107-a44e-4da3-bda8-eb35d1e77692 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:22.120076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae15107-a44e-4da3-bda8-eb35d1e77692 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:22.122580 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae15107-a44e-4da3-bda8-eb35d1e77692 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:22.132876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 339/673] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:22.134275 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae15107-a44e-4da3-bda8-eb35d1e77692 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:22.151402 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:22.162201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256f19f-3ba8-4871-8030-d88e126cc38d None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:22.205535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e062483-34ed-4f55-ae23-5b3855627a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:22.206487 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e062483-34ed-4f55-ae23-5b3855627a1f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:22.206772 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e062483-34ed-4f55-ae23-5b3855627a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:22.207049 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e062483-34ed-4f55-ae23-5b3855627a1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:22.215956 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e062483-34ed-4f55-ae23-5b3855627a1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:22.218260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e062483-34ed-4f55-ae23-5b3855627a1f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:22.235395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-6e062483-34ed-4f55-ae23-5b3855627a1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:22.235395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:22.235395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:22.235395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:22.235395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:22.235395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:22.235395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:22.235395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:22.235395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:22.235395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:22.235395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:22.235395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:22.235395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:22.235395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:22.235395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:22.236499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:22.236499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:22.236499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:22.236499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:22.236499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:22.236499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:22.236499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:22.236499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:22.236499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:22.236499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:22.236499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:22.236499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:22.236499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:22.236499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:22.236499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:22.236499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:22.236499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:22.237513 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:22.237513 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:22.237513 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:22.237513 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:22.237513 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:22.237513 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:22.237513 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:22.237513 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:22.239482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 335/674] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:22.240437 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e062483-34ed-4f55-ae23-5b3855627a1f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:22.257686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:22.260933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae15107-a44e-4da3-bda8-eb35d1e77692 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:22.311813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a379a7-a6df-4b82-beff-7a28d2d3e146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:22.312637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a379a7-a6df-4b82-beff-7a28d2d3e146 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:22.312926 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a379a7-a6df-4b82-beff-7a28d2d3e146 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:22.313248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a379a7-a6df-4b82-beff-7a28d2d3e146 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:22.314283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a379a7-a6df-4b82-beff-7a28d2d3e146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:22.314598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a379a7-a6df-4b82-beff-7a28d2d3e146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:22.316257 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a379a7-a6df-4b82-beff-7a28d2d3e146 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:22.331209 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 340/675] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:22.332358 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a379a7-a6df-4b82-beff-7a28d2d3e146 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:22.363080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e062483-34ed-4f55-ae23-5b3855627a1f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:22.415461 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:22.416454 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc303885-1d09-4fe6-8c69-81cae6a616f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:22.417193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc303885-1d09-4fe6-8c69-81cae6a616f7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:22.417482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc303885-1d09-4fe6-8c69-81cae6a616f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:22.417756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc303885-1d09-4fe6-8c69-81cae6a616f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:22.431484 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc303885-1d09-4fe6-8c69-81cae6a616f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:22.433273 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc303885-1d09-4fe6-8c69-81cae6a616f7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:22.447908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a379a7-a6df-4b82-beff-7a28d2d3e146 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:22.459463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-fc303885-1d09-4fe6-8c69-81cae6a616f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:22.459463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:22.459463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:22.459463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:22.459463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:22.459463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:22.459463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:22.459463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:22.459463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:22.459463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:22.459463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:22.459463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:22.459463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:22.459463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:22.459463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:22.460388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:22.460388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:22.460388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:22.460388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:22.460388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:22.460388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:22.460388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:22.460388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:22.460388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:22.460388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:22.460388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:22.460388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:22.460388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:22.460388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:22.460388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:22.460388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:22.460388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:22.461304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:22.461304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:22.461304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:22.461304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:22.461304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 336/676] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:22.462015 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc303885-1d09-4fe6-8c69-81cae6a616f7 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:22.492765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee2080-2c9c-4f11-b22e-d65adf38e69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:22.493737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee2080-2c9c-4f11-b22e-d65adf38e69e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:22.494024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee2080-2c9c-4f11-b22e-d65adf38e69e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:22.494303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee2080-2c9c-4f11-b22e-d65adf38e69e None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:22.495336 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee2080-2c9c-4f11-b22e-d65adf38e69e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:22.495705 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee2080-2c9c-4f11-b22e-d65adf38e69e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:22.498421 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee2080-2c9c-4f11-b22e-d65adf38e69e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:22.517199 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 341/677] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:22.518459 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee2080-2c9c-4f11-b22e-d65adf38e69e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:22.542405 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:22.587290 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc303885-1d09-4fe6-8c69-81cae6a616f7 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:22.634280 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15243fb1-5485-490b-874b-c021efe0bead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:22.635435 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15243fb1-5485-490b-874b-c021efe0bead None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:22.635935 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15243fb1-5485-490b-874b-c021efe0bead None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:22.637032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15243fb1-5485-490b-874b-c021efe0bead None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:22.647365 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15243fb1-5485-490b-874b-c021efe0bead None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:22.648489 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee2080-2c9c-4f11-b22e-d65adf38e69e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:22.649380 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15243fb1-5485-490b-874b-c021efe0bead None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:22.668048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-15243fb1-5485-490b-874b-c021efe0bead None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:22.668048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:22.668048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:22.668048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:22.668048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:22.668048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:22.668048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:22.668048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:22.668048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:22.668048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:22.668048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:22.668048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:22.668048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:22.668048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:22.668048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:22.668948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:22.668948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:22.668948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:22.668948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:22.668948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:22.668948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:22.668948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:22.668948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:22.668948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:22.668948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:22.668948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:22.668948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:22.668948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:22.668948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:22.668948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:22.668948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:22.668948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:22.670409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:22.670409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:22.670409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:22.670409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:22.670409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:22.670409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:22.670409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:22.670409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:22.671672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 337/678] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:22.672014 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15243fb1-5485-490b-874b-c021efe0bead None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:22.682952 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:22.688703 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b887e7-dadc-499c-8dbb-8716fb87b458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:22.689419 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b887e7-dadc-499c-8dbb-8716fb87b458 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:22.693528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b887e7-dadc-499c-8dbb-8716fb87b458 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:22.693528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b887e7-dadc-499c-8dbb-8716fb87b458 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:22.693528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b887e7-dadc-499c-8dbb-8716fb87b458 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:22.693528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b887e7-dadc-499c-8dbb-8716fb87b458 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:22.693528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b887e7-dadc-499c-8dbb-8716fb87b458 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:22.709136 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 342/679] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:22.709886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b887e7-dadc-499c-8dbb-8716fb87b458 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:22.808125 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15243fb1-5485-490b-874b-c021efe0bead None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:22.824832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b887e7-dadc-499c-8dbb-8716fb87b458 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:22.831763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:22.843372 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ac9ec-03fa-4606-9938-4f16611a7731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:22.844025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ac9ec-03fa-4606-9938-4f16611a7731 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:22.844308 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ac9ec-03fa-4606-9938-4f16611a7731 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:22.844585 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ac9ec-03fa-4606-9938-4f16611a7731 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:22.845442 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ac9ec-03fa-4606-9938-4f16611a7731 None admin] RBAC: Adding query filter params (role.id=a04f2ab5f1234bd6b1dc4c18b4210a89, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=4a87f8bb06ee4a3ba7017422f4f1943e) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:22.845753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ac9ec-03fa-4606-9938-4f16611a7731 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:22.847195 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ac9ec-03fa-4606-9938-4f16611a7731 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:22.853437 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 338/680] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:22 2022] GET /identity/v3/role_assignments?user.id=4a87f8bb06ee4a3ba7017422f4f1943e&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:22.854355 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ac9ec-03fa-4606-9938-4f16611a7731 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:22.873396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816064d-37b2-48b5-8c05-add7a276e3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:22.874502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816064d-37b2-48b5-8c05-add7a276e3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:22.875086 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816064d-37b2-48b5-8c05-add7a276e3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:22.875557 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816064d-37b2-48b5-8c05-add7a276e3b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:22.876568 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816064d-37b2-48b5-8c05-add7a276e3b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:22.877108 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816064d-37b2-48b5-8c05-add7a276e3b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:22.878814 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816064d-37b2-48b5-8c05-add7a276e3b8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:22.894766 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 343/681] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:22.896545 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0192c0-18df-4807-938b-fdc40aa38309 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:22.897092 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0192c0-18df-4807-938b-fdc40aa38309 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:22.897565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0192c0-18df-4807-938b-fdc40aa38309 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:22.900027 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 344/682] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:22.901410 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0192c0-18df-4807-938b-fdc40aa38309 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:22.972210 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ac9ec-03fa-4606-9938-4f16611a7731 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:22.983509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:23.013437 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d35a6-c0d7-4cea-bf9f-02cd1a00fd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:23.014248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d35a6-c0d7-4cea-bf9f-02cd1a00fd1a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:23.014532 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d35a6-c0d7-4cea-bf9f-02cd1a00fd1a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:23.014817 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d35a6-c0d7-4cea-bf9f-02cd1a00fd1a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:23.026176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d35a6-c0d7-4cea-bf9f-02cd1a00fd1a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:23.028041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d35a6-c0d7-4cea-bf9f-02cd1a00fd1a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:23.045951 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0192c0-18df-4807-938b-fdc40aa38309 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:23.047679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-8c4d35a6-c0d7-4cea-bf9f-02cd1a00fd1a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:20:23.047679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:23.047679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:23.047679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:23.047679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:23.047679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:23.047679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:23.047679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:23.047679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:23.047679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:23.047679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:23.047679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:23.047679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:23.047679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:23.047679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:23.047679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:23.048634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:23.048634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:23.048634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:23.048634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:23.048634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:23.048634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:23.048634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:23.048634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:23.048634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:23.048634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:23.048634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:23.048634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:23.048634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:23.048634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:23.048634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:23.048634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:23.048634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:23.048634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:23.049902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:23.049902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:23.049902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:23.049902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:23.049902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:23.049902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:23.049902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:23.049902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:23.049902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:20:23.049902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:23.053100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 339/683] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Jun 7 08:20:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:23.055890 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd87f47-e670-4e42-b6aa-58f9fc62b9a0 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:23.056494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd87f47-e670-4e42-b6aa-58f9fc62b9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:23.056771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd87f47-e670-4e42-b6aa-58f9fc62b9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:23.074571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1fd87f47-e670-4e42-b6aa-58f9fc62b9a0 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:23.075977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1fd87f47-e670-4e42-b6aa-58f9fc62b9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:23.101184 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea208ae7-ade6-438c-bbc1-3fdf46cfa1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:23.102119 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea208ae7-ade6-438c-bbc1-3fdf46cfa1cf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:23.102403 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea208ae7-ade6-438c-bbc1-3fdf46cfa1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:23.102676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea208ae7-ade6-438c-bbc1-3fdf46cfa1cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:23.104043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea208ae7-ade6-438c-bbc1-3fdf46cfa1cf None admin] RBAC: Adding query filter params (role.id=a04f2ab5f1234bd6b1dc4c18b4210a89, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=85ea2962e7094f8c878d95680764c4e6) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:23.104366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea208ae7-ade6-438c-bbc1-3fdf46cfa1cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:23.105968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea208ae7-ade6-438c-bbc1-3fdf46cfa1cf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:23.112942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 345/684] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:22 2022] GET /identity/v3/role_assignments?user.id=85ea2962e7094f8c878d95680764c4e6&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:23.113812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea208ae7-ade6-438c-bbc1-3fdf46cfa1cf None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:23.137431 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:23.181232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 340/685] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:23.212434 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd87f47-e670-4e42-b6aa-58f9fc62b9a0 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:23.253683 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea208ae7-ade6-438c-bbc1-3fdf46cfa1cf None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:23.277655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:23.304763 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e07e9-62f5-4b2b-9653-b38e6f4e8108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:23.305726 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e07e9-62f5-4b2b-9653-b38e6f4e8108 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:23.308063 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e07e9-62f5-4b2b-9653-b38e6f4e8108 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:23.308496 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e07e9-62f5-4b2b-9653-b38e6f4e8108 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:23.309835 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e07e9-62f5-4b2b-9653-b38e6f4e8108 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:23.310340 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e07e9-62f5-4b2b-9653-b38e6f4e8108 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:23.313320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e07e9-62f5-4b2b-9653-b38e6f4e8108 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:23.340954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 346/686] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Jun 7 08:20:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:23.344243 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e07e9-62f5-4b2b-9653-b38e6f4e8108 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:23.375058 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd87f47-e670-4e42-b6aa-58f9fc62b9a0 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:23.421999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88860d-b676-4803-852c-8177e0f65210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:23.422820 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88860d-b676-4803-852c-8177e0f65210 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:23.423202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88860d-b676-4803-852c-8177e0f65210 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:23.423580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88860d-b676-4803-852c-8177e0f65210 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:23.437660 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e88860d-b676-4803-852c-8177e0f65210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:23.446275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-9e88860d-b676-4803-852c-8177e0f65210 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:23.446275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:23.446275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:23.446275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:23.446275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:23.446275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:23.446275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:23.446275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:23.446275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:23.446275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:23.446275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:23.446275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:23.446275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:23.446275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:23.446275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:23.446275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:23.447494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:23.447494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:23.447494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:23.447494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:23.447494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:23.447494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:23.447494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:23.447494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:23.447494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:23.447494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:23.447494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:23.447494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:23.447494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:23.447494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:23.447494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:23.447494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:23.448535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:23.448535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:23.448535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:23.448535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:23.448535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:23.448535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:23.448535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:23.448535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:23.448535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:23.448535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:23.448535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:23.448535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:23.455665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 341/687] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:23.461069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88860d-b676-4803-852c-8177e0f65210 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:23.484498 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e07e9-62f5-4b2b-9653-b38e6f4e8108 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:23.506333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:23.521970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e576ca0-4ad1-46e1-9b73-4d77d921595e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:23.522864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e576ca0-4ad1-46e1-9b73-4d77d921595e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:23.523353 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e576ca0-4ad1-46e1-9b73-4d77d921595e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:23.523785 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e576ca0-4ad1-46e1-9b73-4d77d921595e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:23.574309 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:23.597552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88860d-b676-4803-852c-8177e0f65210 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:23.608066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e576ca0-4ad1-46e1-9b73-4d77d921595e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:23.610416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e576ca0-4ad1-46e1-9b73-4d77d921595e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:23.637340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7bac5-224f-44f3-90ff-c57271996414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:23.638851 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7bac5-224f-44f3-90ff-c57271996414 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:23.639312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7bac5-224f-44f3-90ff-c57271996414 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:23.639741 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7bac5-224f-44f3-90ff-c57271996414 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:23.640658 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d7bac5-224f-44f3-90ff-c57271996414 None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:23.641186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d7bac5-224f-44f3-90ff-c57271996414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:23.642774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d7bac5-224f-44f3-90ff-c57271996414 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:23.656803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 342/688] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:20:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:23.658350 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7bac5-224f-44f3-90ff-c57271996414 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:23.732133 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 347/689] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 08:20:23.739947 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e576ca0-4ad1-46e1-9b73-4d77d921595e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:23.802982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:23.807388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7bac5-224f-44f3-90ff-c57271996414 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:23.834845 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:23.843446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d1a65-cdd5-4458-986a-8c4a43f90777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:23.844796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d1a65-cdd5-4458-986a-8c4a43f90777 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:23.845272 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d1a65-cdd5-4458-986a-8c4a43f90777 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:23.845775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d1a65-cdd5-4458-986a-8c4a43f90777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:23.857783 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d1a65-cdd5-4458-986a-8c4a43f90777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:23.859637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d1a65-cdd5-4458-986a-8c4a43f90777 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:23.865822 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e576ca0-4ad1-46e1-9b73-4d77d921595e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:23.878891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-ec9d1a65-cdd5-4458-986a-8c4a43f90777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:23.878891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:23.878891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:23.878891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:23.878891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:23.878891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:23.878891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:23.878891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:23.878891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:23.878891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:23.878891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:23.878891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:23.878891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:23.878891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:23.878891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:23.879832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:23.879832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:23.879832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:23.879832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:23.879832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:23.879832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:23.879832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:23.879832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:23.879832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:23.879832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:23.879832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:23.879832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:23.879832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:23.879832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:23.879832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:23.879832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:23.879832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:23.880711 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:23.880711 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:23.880711 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:23.880711 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:23.887187 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 343/690] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:23.889770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12609075-a57f-412e-937c-0cdbf8020eaf None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:23.890274 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12609075-a57f-412e-937c-0cdbf8020eaf None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:23.890771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12609075-a57f-412e-937c-0cdbf8020eaf None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:23.896884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 344/691] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:23.898054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12609075-a57f-412e-937c-0cdbf8020eaf None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:23.904150 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db89847-02a1-47c2-affb-8e642285cc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:23.904952 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db89847-02a1-47c2-affb-8e642285cc67 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:23.905273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db89847-02a1-47c2-affb-8e642285cc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:23.905547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db89847-02a1-47c2-affb-8e642285cc67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:23.917328 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db89847-02a1-47c2-affb-8e642285cc67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:23.919057 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db89847-02a1-47c2-affb-8e642285cc67 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:23.938647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-7db89847-02a1-47c2-affb-8e642285cc67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:23.938647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:23.938647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:23.938647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:23.938647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:23.938647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:23.938647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:23.938647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:23.938647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:23.938647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:23.938647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:23.938647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:23.938647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:23.938647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:23.938647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:23.939562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:23.939562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:23.939562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:23.939562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:23.939562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:23.939562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:23.939562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:23.939562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:23.939562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:23.939562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:23.939562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:23.939562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:23.939562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:23.939562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:23.939562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:23.939562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:23.939562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:23.940526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:23.940526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:23.940526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:23.940526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:23.947204 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 348/692] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:23.948875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222dfac-f923-4ff0-8657-3a53c48ecf06 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:23.949427 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222dfac-f923-4ff0-8657-3a53c48ecf06 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:23.949911 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222dfac-f923-4ff0-8657-3a53c48ecf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:23.969335 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2222dfac-f923-4ff0-8657-3a53c48ecf06 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:23.970970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2222dfac-f923-4ff0-8657-3a53c48ecf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:24.033045 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12609075-a57f-412e-937c-0cdbf8020eaf None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:24.055534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 349/693] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:24.055534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222dfac-f923-4ff0-8657-3a53c48ecf06 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:24.082627 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e497312-311b-455d-a39b-4d12c4003ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:24.083286 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e497312-311b-455d-a39b-4d12c4003ade None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:24.083565 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e497312-311b-455d-a39b-4d12c4003ade None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:24.083837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e497312-311b-455d-a39b-4d12c4003ade None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:24.084555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e497312-311b-455d-a39b-4d12c4003ade None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:24.084861 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e497312-311b-455d-a39b-4d12c4003ade None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:24.088527 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e497312-311b-455d-a39b-4d12c4003ade None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:24.102175 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 345/694] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:24.103476 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e497312-311b-455d-a39b-4d12c4003ade None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:24.116634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:24.127418 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:24.170624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222dfac-f923-4ff0-8657-3a53c48ecf06 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:24.207001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b19b5-7b50-47b1-94e1-3a05f211133a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:24.207674 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b19b5-7b50-47b1-94e1-3a05f211133a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:24.207963 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b19b5-7b50-47b1-94e1-3a05f211133a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:24.208205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b19b5-7b50-47b1-94e1-3a05f211133a None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:24.208914 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b19b5-7b50-47b1-94e1-3a05f211133a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:24.209243 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b19b5-7b50-47b1-94e1-3a05f211133a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:24.210671 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b19b5-7b50-47b1-94e1-3a05f211133a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:24.220489 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e497312-311b-455d-a39b-4d12c4003ade None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:24.222516 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 350/695] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:24.226474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b19b5-7b50-47b1-94e1-3a05f211133a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:24.261107 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9ca20-32d1-4045-8279-217853121e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:24.262356 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9ca20-32d1-4045-8279-217853121e8a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:24.262833 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9ca20-32d1-4045-8279-217853121e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:24.263321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9ca20-32d1-4045-8279-217853121e8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:24.274892 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9ca20-32d1-4045-8279-217853121e8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:24.276725 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9ca20-32d1-4045-8279-217853121e8a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:24.296747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-b8b9ca20-32d1-4045-8279-217853121e8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:24.296747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:24.296747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:24.296747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:24.296747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:24.296747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:24.296747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:24.296747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:24.296747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:24.296747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:24.296747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:24.296747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:24.296747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:24.296747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:24.296747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:24.297702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:24.297702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:24.297702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:24.297702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:24.297702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:24.297702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:24.297702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:24.297702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:24.297702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:24.297702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:24.297702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:24.297702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:24.297702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:24.297702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:24.297702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:24.297702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:24.297702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:24.298617 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:24.298617 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:24.298617 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:24.298617 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:24.305641 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 346/696] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:24.306793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9ca20-32d1-4045-8279-217853121e8a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:24.329021 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b19b5-7b50-47b1-94e1-3a05f211133a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:24.346532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:24.364135 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3af4c-0d9e-45e8-ad1a-076c7415e616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:24.365105 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3af4c-0d9e-45e8-ad1a-076c7415e616 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:24.365497 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3af4c-0d9e-45e8-ad1a-076c7415e616 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:24.365844 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3af4c-0d9e-45e8-ad1a-076c7415e616 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:24.374406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab3af4c-0d9e-45e8-ad1a-076c7415e616 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:24.375971 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab3af4c-0d9e-45e8-ad1a-076c7415e616 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:24.378681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:24.391044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1ab3af4c-0d9e-45e8-ad1a-076c7415e616 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:20:24.391044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:24.391044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:24.391044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:24.391044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:24.391044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:24.391044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:24.391044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:24.391044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:24.391044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:24.391044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:24.391044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:24.391044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:24.391044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:24.391044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:24.391044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:24.393480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:24.393480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:24.393480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:24.393480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:24.393480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:24.393480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:24.393480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:24.393480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:24.393480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:24.393480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:24.393480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:24.393480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:24.393480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:24.393480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:24.393480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:24.393480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:24.393480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:24.393480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:24.394424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:24.394424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:24.394424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:24.394424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:24.394424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:24.394424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:24.394424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:24.394424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:24.394424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:20:24.394424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:24.394424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 351/697] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:20:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:24.394424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3af4c-0d9e-45e8-ad1a-076c7415e616 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:24.423012 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9ca20-32d1-4045-8279-217853121e8a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:24.462352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac89fb5-c293-4d78-8db9-78dd33fb5312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:24.463406 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac89fb5-c293-4d78-8db9-78dd33fb5312 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:24.463835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac89fb5-c293-4d78-8db9-78dd33fb5312 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:24.464259 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac89fb5-c293-4d78-8db9-78dd33fb5312 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:24.471908 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac89fb5-c293-4d78-8db9-78dd33fb5312 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:24.473679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac89fb5-c293-4d78-8db9-78dd33fb5312 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:24.488769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-eac89fb5-c293-4d78-8db9-78dd33fb5312 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:24.488769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:24.488769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:24.488769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:24.488769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:24.488769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:24.488769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:24.488769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:24.488769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:24.488769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:24.488769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:24.488769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:24.488769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:24.488769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:24.488769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:24.489934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:24.489934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:24.489934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:24.489934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:24.489934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:24.489934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:24.489934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:24.489934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:24.489934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:24.489934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:24.489934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:24.489934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:24.489934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:24.489934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:24.489934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:24.489934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:24.489934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:24.490806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:24.490806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:24.490806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:24.490806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:24.490806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:24.490806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:24.490806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:24.490806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:24.493239 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 347/698] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:24.494452 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac89fb5-c293-4d78-8db9-78dd33fb5312 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:24.497186 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3af4c-0d9e-45e8-ad1a-076c7415e616 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:24.531458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412334a1-af7a-4ec0-9de9-3dfb250862be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:24.532115 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412334a1-af7a-4ec0-9de9-3dfb250862be None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:24.533087 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412334a1-af7a-4ec0-9de9-3dfb250862be None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:24.533087 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412334a1-af7a-4ec0-9de9-3dfb250862be None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:24.533385 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412334a1-af7a-4ec0-9de9-3dfb250862be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:24.533695 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412334a1-af7a-4ec0-9de9-3dfb250862be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:24.535117 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412334a1-af7a-4ec0-9de9-3dfb250862be None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:24.553362 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 352/699] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:24.553362 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412334a1-af7a-4ec0-9de9-3dfb250862be None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:24.589174 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:24.609754 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac89fb5-c293-4d78-8db9-78dd33fb5312 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:24.633923 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:24.646916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31114eac-7a5f-4d0a-b4e2-7679a3cf9413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:24.648159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31114eac-7a5f-4d0a-b4e2-7679a3cf9413 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:24.648617 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31114eac-7a5f-4d0a-b4e2-7679a3cf9413 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:24.649106 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31114eac-7a5f-4d0a-b4e2-7679a3cf9413 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:24.650265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31114eac-7a5f-4d0a-b4e2-7679a3cf9413 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:24.650861 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31114eac-7a5f-4d0a-b4e2-7679a3cf9413 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:24.652767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31114eac-7a5f-4d0a-b4e2-7679a3cf9413 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:24.654925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412334a1-af7a-4ec0-9de9-3dfb250862be None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:24.678559 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 348/700] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Jun 7 08:20:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:24.679728 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31114eac-7a5f-4d0a-b4e2-7679a3cf9413 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:24.691464 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a5832-7f43-4e64-9a9c-ad9dc9eb8b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:24.692126 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a5832-7f43-4e64-9a9c-ad9dc9eb8b2f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:24.692402 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a5832-7f43-4e64-9a9c-ad9dc9eb8b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:24.692669 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a5832-7f43-4e64-9a9c-ad9dc9eb8b2f None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:24.693492 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a5832-7f43-4e64-9a9c-ad9dc9eb8b2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:24.693799 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a5832-7f43-4e64-9a9c-ad9dc9eb8b2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:24.695392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a5832-7f43-4e64-9a9c-ad9dc9eb8b2f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:24.707241 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 353/701] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:24.708065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a5832-7f43-4e64-9a9c-ad9dc9eb8b2f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:24.793435 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31114eac-7a5f-4d0a-b4e2-7679a3cf9413 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:24.803425 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:24.809018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a5832-7f43-4e64-9a9c-ad9dc9eb8b2f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:24.832240 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7e82b-991b-40f6-a266-dd81fd9c3d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:24.833317 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7e82b-991b-40f6-a266-dd81fd9c3d22 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:24.833803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7e82b-991b-40f6-a266-dd81fd9c3d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:24.834343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7e82b-991b-40f6-a266-dd81fd9c3d22 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:24.835249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7e82b-991b-40f6-a266-dd81fd9c3d22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:24.836984 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7e82b-991b-40f6-a266-dd81fd9c3d22 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:24.848733 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22952f7a-6e99-4119-bbe5-bf8bf4884fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:24.849899 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22952f7a-6e99-4119-bbe5-bf8bf4884fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:24.850270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22952f7a-6e99-4119-bbe5-bf8bf4884fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:24.850683 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22952f7a-6e99-4119-bbe5-bf8bf4884fe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:24.907766 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 349/702] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jun 7 08:20:24 2022] POST /identity/v3/users => generated 254 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:24.909529 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7e82b-991b-40f6-a266-dd81fd9c3d22 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:24.930918 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22952f7a-6e99-4119-bbe5-bf8bf4884fe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:24.932930 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22952f7a-6e99-4119-bbe5-bf8bf4884fe5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:24.990109 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:25.049491 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:25.058393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 354/703] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:20:25.063982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22952f7a-6e99-4119-bbe5-bf8bf4884fe5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:25.160888 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7e82b-991b-40f6-a266-dd81fd9c3d22 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:25.219398 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659ba77-110b-45bb-a157-419c4aa7b68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:25.220144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659ba77-110b-45bb-a157-419c4aa7b68f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:25.220518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659ba77-110b-45bb-a157-419c4aa7b68f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:25.220865 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659ba77-110b-45bb-a157-419c4aa7b68f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:25.221879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659ba77-110b-45bb-a157-419c4aa7b68f None admin] RBAC: Adding query filter params (role.id=8c63c00418a346b29f7db3f87a404e70, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=ee42568b059b49a198ed2c01d5efdafd) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:25.222259 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659ba77-110b-45bb-a157-419c4aa7b68f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:25.223772 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659ba77-110b-45bb-a157-419c4aa7b68f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:25.230750 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 350/704] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:24 2022] GET /identity/v3/role_assignments?user.id=ee42568b059b49a198ed2c01d5efdafd&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 259 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:25.241410 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22952f7a-6e99-4119-bbe5-bf8bf4884fe5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:25.281496 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294cfb41-e343-44f0-a8fe-b6feebf027b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:25.282493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cfb41-e343-44f0-a8fe-b6feebf027b2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:25.282876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cfb41-e343-44f0-a8fe-b6feebf027b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:25.283254 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cfb41-e343-44f0-a8fe-b6feebf027b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:25.294823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294cfb41-e343-44f0-a8fe-b6feebf027b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:25.296468 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294cfb41-e343-44f0-a8fe-b6feebf027b2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:25.318812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-294cfb41-e343-44f0-a8fe-b6feebf027b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:25.318812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:25.318812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:25.318812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:25.318812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:25.318812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:25.318812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:25.318812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:25.318812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:25.318812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:25.318812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:25.318812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:25.318812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:25.318812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:25.318812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:25.319927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:25.319927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:25.319927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:25.319927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:25.319927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:25.319927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:25.319927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:25.319927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:25.319927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:25.319927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:25.319927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:25.319927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:25.319927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:25.319927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:25.319927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:25.319927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:25.319927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:25.321032 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:25.321032 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:25.321032 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:25.321032 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:25.322822 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 355/705] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:25.330139 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659ba77-110b-45bb-a157-419c4aa7b68f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:25.444146 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:25.447699 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659ba77-110b-45bb-a157-419c4aa7b68f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:25.469929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04f913-3b55-469c-8d5b-6c7a79c5b903 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:25.470443 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04f913-3b55-469c-8d5b-6c7a79c5b903 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:25.470698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04f913-3b55-469c-8d5b-6c7a79c5b903 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:25.477793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 356/706] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:20:25.479852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7d851-59bf-4578-a4fa-5beb5619ee6b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:25.480261 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7d851-59bf-4578-a4fa-5beb5619ee6b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:25.481608 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7d851-59bf-4578-a4fa-5beb5619ee6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:25.504684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432d591-a224-40dd-9c88-af81a38f9044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:25.505420 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432d591-a224-40dd-9c88-af81a38f9044 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:25.505701 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432d591-a224-40dd-9c88-af81a38f9044 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:25.505972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432d591-a224-40dd-9c88-af81a38f9044 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:25.506705 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432d591-a224-40dd-9c88-af81a38f9044 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:25.507010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432d591-a224-40dd-9c88-af81a38f9044 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:25.508392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432d591-a224-40dd-9c88-af81a38f9044 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:25.521203 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9ae7d851-59bf-4578-a4fa-5beb5619ee6b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:25.522476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9ae7d851-59bf-4578-a4fa-5beb5619ee6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:25.537469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 351/707] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:25.547791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432d591-a224-40dd-9c88-af81a38f9044 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:25.615581 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:25.660274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 357/708] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:25.680859 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7d851-59bf-4578-a4fa-5beb5619ee6b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:25.723970 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432d591-a224-40dd-9c88-af81a38f9044 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:25.760173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1b829-0e5c-4c7f-ae42-3160d331aeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:25.761097 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1b829-0e5c-4c7f-ae42-3160d331aeb9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:25.761455 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1b829-0e5c-4c7f-ae42-3160d331aeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:25.761804 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1b829-0e5c-4c7f-ae42-3160d331aeb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:25.771661 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1b829-0e5c-4c7f-ae42-3160d331aeb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:25.773230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1b829-0e5c-4c7f-ae42-3160d331aeb9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:25.796789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-91d1b829-0e5c-4c7f-ae42-3160d331aeb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:25.796789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:25.796789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:25.796789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:25.796789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:25.796789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:25.796789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:25.796789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:25.796789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:25.796789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:25.796789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:25.796789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:25.796789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:25.796789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:25.796789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:25.797978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:25.797978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:25.797978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:25.797978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:25.797978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:25.797978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:25.797978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:25.797978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:25.797978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:25.797978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:25.797978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:25.797978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:25.797978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:25.797978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:25.797978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:25.797978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:25.797978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:25.799362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:25.799362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:25.799362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:25.799362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:25.799362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:25.799362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:25.799362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:25.799362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:25.815259 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 352/709] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:25.816535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1b829-0e5c-4c7f-ae42-3160d331aeb9 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:25.836371 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7d851-59bf-4578-a4fa-5beb5619ee6b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:25.861679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:25.877952 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc0c33-5fec-42a9-b825-14a8b2904fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:25.879148 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc0c33-5fec-42a9-b825-14a8b2904fed None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:25.879648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc0c33-5fec-42a9-b825-14a8b2904fed None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:25.880085 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc0c33-5fec-42a9-b825-14a8b2904fed None admin] PATH_INFO: `/v3/services/network` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:25.880942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc0c33-5fec-42a9-b825-14a8b2904fed None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:25.882649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc0c33-5fec-42a9-b825-14a8b2904fed None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:25.897018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-debc0c33-5fec-42a9-b825-14a8b2904fed None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 08:20:25.897018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:25.897018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:20:25.897018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:20:25.897018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:20:25.897018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:20:25.897018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:20:25.897018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:25.897018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 08:20:25.897018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:25.897018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:25.897018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:25.897018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:25.897018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:25.897018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:25.897018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:25.897018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:25.898097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:25.898097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:25.898097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:25.898097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:25.898097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:25.898097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:25.898097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:20:25.898097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:20:25.898097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:20:25.898097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:20:25.898097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:25.898097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:25.898097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:25.898097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:25.898097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:25.898097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:25.898925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:25.898925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:25.898925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:25.898925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:25.898925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:25.898925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:25.898925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:25.898925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:25.898925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:25.898925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:25.898925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:20:25.898925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:25.898925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 08:20:25.898925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:25.901842 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 358/710] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:25 2022] GET /identity/v3/services/network => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:25.903299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c90378-f649-4929-9691-ab913cc01895 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:25.903923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c90378-f649-4929-9691-ab913cc01895 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:25.904360 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c90378-f649-4929-9691-ab913cc01895 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:25.906024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 359/711] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:25.908827 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c90378-f649-4929-9691-ab913cc01895 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:25.968214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1b829-0e5c-4c7f-ae42-3160d331aeb9 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:25.969638 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:26.012866 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc762be-40d7-4f41-99d3-ad65a6978042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:26.014365 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc762be-40d7-4f41-99d3-ad65a6978042 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:26.014745 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc762be-40d7-4f41-99d3-ad65a6978042 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:26.015174 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc762be-40d7-4f41-99d3-ad65a6978042 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:26.015993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc762be-40d7-4f41-99d3-ad65a6978042 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:26.016414 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc762be-40d7-4f41-99d3-ad65a6978042 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:26.018172 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc762be-40d7-4f41-99d3-ad65a6978042 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:26.031209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 353/712] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:26.032744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fc0fe-a89c-4528-be3b-df97072b54e1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:26.033183 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fc0fe-a89c-4528-be3b-df97072b54e1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:26.033551 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fc0fe-a89c-4528-be3b-df97072b54e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:26.035149 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c90378-f649-4929-9691-ab913cc01895 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:26.051111 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-007fc0fe-a89c-4528-be3b-df97072b54e1 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:26.052435 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-007fc0fe-a89c-4528-be3b-df97072b54e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:26.072717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e78f9f-df53-4d73-beba-22d2fa4f45af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:26.073640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e78f9f-df53-4d73-beba-22d2fa4f45af None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:26.074117 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e78f9f-df53-4d73-beba-22d2fa4f45af None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:26.074560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e78f9f-df53-4d73-beba-22d2fa4f45af None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:26.075405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e78f9f-df53-4d73-beba-22d2fa4f45af None admin] RBAC: Adding query filter params (name=network) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:26.076425 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e78f9f-df53-4d73-beba-22d2fa4f45af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:26.078059 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e78f9f-df53-4d73-beba-22d2fa4f45af None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:26.087614 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 360/713] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jun 7 08:20:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:26.088690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e78f9f-df53-4d73-beba-22d2fa4f45af None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:26.126117 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 354/714] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:26.127151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fc0fe-a89c-4528-be3b-df97072b54e1 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:26.140175 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:26.194147 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e78f9f-df53-4d73-beba-22d2fa4f45af None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:26.202715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:26.230858 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b6599-e173-462c-a754-6cd6042d2cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:26.231433 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b6599-e173-462c-a754-6cd6042d2cec None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:26.231715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b6599-e173-462c-a754-6cd6042d2cec None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:26.232019 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b6599-e173-462c-a754-6cd6042d2cec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:26.232851 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b6599-e173-462c-a754-6cd6042d2cec None admin] RBAC: Adding query filter params (role.id=a04f2ab5f1234bd6b1dc4c18b4210a89, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=334dc47ae2e643ccb44d25c6b5906f14) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:26.233207 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b6599-e173-462c-a754-6cd6042d2cec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:26.234636 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b6599-e173-462c-a754-6cd6042d2cec None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:26.236170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fc0fe-a89c-4528-be3b-df97072b54e1 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:26.241949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 361/715] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:26 2022] GET /identity/v3/role_assignments?user.id=334dc47ae2e643ccb44d25c6b5906f14&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:26.242770 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b6599-e173-462c-a754-6cd6042d2cec None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:26.277756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221aef51-68f4-43ef-a661-e2760a2d16a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:26.278980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221aef51-68f4-43ef-a661-e2760a2d16a7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:26.279509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221aef51-68f4-43ef-a661-e2760a2d16a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:26.280051 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221aef51-68f4-43ef-a661-e2760a2d16a7 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:26.281117 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221aef51-68f4-43ef-a661-e2760a2d16a7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:26.281655 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221aef51-68f4-43ef-a661-e2760a2d16a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:26.283235 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221aef51-68f4-43ef-a661-e2760a2d16a7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:26.290344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 355/716] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jun 7 08:20:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:26.408003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b6599-e173-462c-a754-6cd6042d2cec None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:26.467501 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12abe04-d533-412b-a6bb-dad2e1ca5ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:26.468346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12abe04-d533-412b-a6bb-dad2e1ca5ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:26.468630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12abe04-d533-412b-a6bb-dad2e1ca5ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:26.468906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12abe04-d533-412b-a6bb-dad2e1ca5ec7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:26.469608 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12abe04-d533-412b-a6bb-dad2e1ca5ec7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:26.471029 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12abe04-d533-412b-a6bb-dad2e1ca5ec7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:26.490992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-f12abe04-d533-412b-a6bb-dad2e1ca5ec7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 08:20:26.490992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:26.490992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:20:26.490992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:20:26.490992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:20:26.490992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:20:26.490992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:20:26.490992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:26.490992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 08:20:26.490992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:26.490992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:26.490992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:26.490992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:26.490992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:26.490992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:26.490992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:26.492004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:26.492004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:26.492004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:26.492004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:26.492004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:26.492004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:26.492004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:26.492004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:20:26.492004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:20:26.492004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:20:26.492004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:20:26.492004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:26.492004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:26.492004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:26.492004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:26.492004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:26.492823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:26.492823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:26.492823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:26.492823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:26.492823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:26.492823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:26.492823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:26.492823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:26.492823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:26.492823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:26.492823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:26.492823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:20:26.492823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:26.492823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 08:20:26.492823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:26.497172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 362/717] 10.222.0.17 () {56 vars in 1129 bytes} [Tue Jun 7 08:20:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:26.499617 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221aef51-68f4-43ef-a661-e2760a2d16a7 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:26.581193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:26.609640 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221aef51-68f4-43ef-a661-e2760a2d16a7 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:26.648378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350e6de-34d9-4522-a32c-31fb781c5442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:26.649249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350e6de-34d9-4522-a32c-31fb781c5442 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:26.649627 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350e6de-34d9-4522-a32c-31fb781c5442 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:26.650000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350e6de-34d9-4522-a32c-31fb781c5442 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:26.650808 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0350e6de-34d9-4522-a32c-31fb781c5442 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:26.651210 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0350e6de-34d9-4522-a32c-31fb781c5442 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:26.652671 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0350e6de-34d9-4522-a32c-31fb781c5442 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:26.658113 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 356/718] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Jun 7 08:20:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:26.663479 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12abe04-d533-412b-a6bb-dad2e1ca5ec7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:26.703796 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:26.779288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12abe04-d533-412b-a6bb-dad2e1ca5ec7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:26.820009 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db20ed-9a10-4fe1-a233-638b05b93466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:26.820682 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db20ed-9a10-4fe1-a233-638b05b93466 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:26.820961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db20ed-9a10-4fe1-a233-638b05b93466 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:26.821266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db20ed-9a10-4fe1-a233-638b05b93466 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:26.821978 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db20ed-9a10-4fe1-a233-638b05b93466 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:26.822278 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db20ed-9a10-4fe1-a233-638b05b93466 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:26.823648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db20ed-9a10-4fe1-a233-638b05b93466 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:26.830729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 363/719] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Jun 7 08:20:26 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:27.828472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbbb22-39a0-4529-8ec1-9fe0b37dc28c None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:27.829022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbbb22-39a0-4529-8ec1-9fe0b37dc28c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:27.829135 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbbb22-39a0-4529-8ec1-9fe0b37dc28c None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:27.830883 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 357/720] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:27.835870 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad373d03-5d36-4026-a57d-5367d0424828 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:27.836263 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad373d03-5d36-4026-a57d-5367d0424828 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:27.836434 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad373d03-5d36-4026-a57d-5367d0424828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:27.858212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ad373d03-5d36-4026-a57d-5367d0424828 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:27.859419 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ad373d03-5d36-4026-a57d-5367d0424828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:27.898736 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576209f-c9d4-41d3-9e1c-bcedcb03c406 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:27.899042 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576209f-c9d4-41d3-9e1c-bcedcb03c406 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:27.899295 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576209f-c9d4-41d3-9e1c-bcedcb03c406 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:27.900940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 358/721] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:27.905653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec4652-fbc7-452b-9c4b-016ecaef86b4 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:27.905936 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec4652-fbc7-452b-9c4b-016ecaef86b4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:27.906210 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec4652-fbc7-452b-9c4b-016ecaef86b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:27.932718 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a6ec4652-fbc7-452b-9c4b-016ecaef86b4 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:27.933941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a6ec4652-fbc7-452b-9c4b-016ecaef86b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:27.961221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 364/722] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:28.007192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad373d03-5d36-4026-a57d-5367d0424828 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:28.008030 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 359/723] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 08:20:28.041797 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec4652-fbc7-452b-9c4b-016ecaef86b4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:28.062890 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:28.153524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad373d03-5d36-4026-a57d-5367d0424828 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:28.202425 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde7342-3c6b-4c30-900b-4bc02c7fff92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:28.203231 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde7342-3c6b-4c30-900b-4bc02c7fff92 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:28.203509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde7342-3c6b-4c30-900b-4bc02c7fff92 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:28.203781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde7342-3c6b-4c30-900b-4bc02c7fff92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:28.210099 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:28.228951 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde7342-3c6b-4c30-900b-4bc02c7fff92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:28.238772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-cbde7342-3c6b-4c30-900b-4bc02c7fff92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:28.238772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:28.238772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:28.238772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:28.238772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:28.238772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:28.238772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:28.238772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:28.238772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:28.238772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:28.238772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:28.238772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:28.238772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:28.238772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:28.238772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:28.238772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:28.239729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:28.239729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:28.239729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:28.239729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:28.239729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:28.239729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:28.239729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:28.239729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:28.239729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:28.239729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:28.239729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:28.239729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:28.239729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:28.239729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:28.239729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:28.239729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:28.240565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:28.240565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:28.240565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:28.240565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:28.240565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:28.240565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:28.240565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:28.240565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:28.240565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:28.240565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:28.240565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:28.240565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:28.252124 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 365/724] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:28.253025 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde7342-3c6b-4c30-900b-4bc02c7fff92 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:28.269920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec4652-fbc7-452b-9c4b-016ecaef86b4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:28.386239 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715b78a-0ef2-4c3b-a63b-413787a3470c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:28.387040 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715b78a-0ef2-4c3b-a63b-413787a3470c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:28.387319 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715b78a-0ef2-4c3b-a63b-413787a3470c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:28.387594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715b78a-0ef2-4c3b-a63b-413787a3470c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:28.407965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:28.420958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde7342-3c6b-4c30-900b-4bc02c7fff92 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:28.421921 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715b78a-0ef2-4c3b-a63b-413787a3470c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:28.438267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-9715b78a-0ef2-4c3b-a63b-413787a3470c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:28.438267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:28.438267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:28.438267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:28.438267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:28.438267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:28.438267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:28.438267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:28.438267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:28.438267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:28.438267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:28.438267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:28.438267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:28.438267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:28.438267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:28.438267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:28.440418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:28.440418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:28.440418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:28.440418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:28.440418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:28.440418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:28.440418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:28.440418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:28.440418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:28.440418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:28.440418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:28.440418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:28.440418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:28.440418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:28.440418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:28.440418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:28.442592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:28.442592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:28.442592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:28.442592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:28.442592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:28.442592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:28.442592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:28.442592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:28.442592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:28.442592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:28.442592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:28.442592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:28.442592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 360/725] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:20:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:28.449106 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715b78a-0ef2-4c3b-a63b-413787a3470c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:28.469840 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c52da-fbf5-4b9c-a8c1-7dda383a9d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:28.470532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c52da-fbf5-4b9c-a8c1-7dda383a9d40 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:28.470816 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c52da-fbf5-4b9c-a8c1-7dda383a9d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:28.471087 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c52da-fbf5-4b9c-a8c1-7dda383a9d40 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:28.471843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c52da-fbf5-4b9c-a8c1-7dda383a9d40 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:28.472149 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c52da-fbf5-4b9c-a8c1-7dda383a9d40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:28.473616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c52da-fbf5-4b9c-a8c1-7dda383a9d40 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:28.487114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 366/726] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:20:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:28.494302 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c52da-fbf5-4b9c-a8c1-7dda383a9d40 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:28.563304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715b78a-0ef2-4c3b-a63b-413787a3470c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:28.590995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:28.613622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce2438-a296-46b0-a048-811298a61560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:28.614317 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce2438-a296-46b0-a048-811298a61560 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:28.614600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce2438-a296-46b0-a048-811298a61560 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:28.614877 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce2438-a296-46b0-a048-811298a61560 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:28.615656 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce2438-a296-46b0-a048-811298a61560 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:28.615964 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce2438-a296-46b0-a048-811298a61560 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:28.617446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce2438-a296-46b0-a048-811298a61560 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:28.617947 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c52da-fbf5-4b9c-a8c1-7dda383a9d40 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:28.636300 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 361/727] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:28.641727 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce2438-a296-46b0-a048-811298a61560 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:28.666329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c3a9d-506b-47d9-8f5e-895703afde89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:28.667255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c3a9d-506b-47d9-8f5e-895703afde89 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:28.667439 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c3a9d-506b-47d9-8f5e-895703afde89 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:28.667761 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c3a9d-506b-47d9-8f5e-895703afde89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:28.681046 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c3a9d-506b-47d9-8f5e-895703afde89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:28.682512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c3a9d-506b-47d9-8f5e-895703afde89 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:28.700927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-649c3a9d-506b-47d9-8f5e-895703afde89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:28.700927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:28.700927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:28.700927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:28.700927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:28.700927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:28.700927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:28.700927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:28.700927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:28.700927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:28.700927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:28.700927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:28.700927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:28.700927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:28.700927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:28.703233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:28.703233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:28.703233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:28.703233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:28.703233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:28.703233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:28.703233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:28.703233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:28.703233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:28.703233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:28.703233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:28.703233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:28.703233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:28.703233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:28.703233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:28.703233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:28.703233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:28.705466 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:28.705466 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:28.705466 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:28.705466 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:28.705466 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 367/728] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:28.710032 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c3a9d-506b-47d9-8f5e-895703afde89 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:28.715683 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:28.802354 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce2438-a296-46b0-a048-811298a61560 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:28.873713 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ce16a-77b9-4c17-aae9-6cebbe0e8321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:28.874519 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ce16a-77b9-4c17-aae9-6cebbe0e8321 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:28.874806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ce16a-77b9-4c17-aae9-6cebbe0e8321 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:28.875085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ce16a-77b9-4c17-aae9-6cebbe0e8321 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:28.877965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c3a9d-506b-47d9-8f5e-895703afde89 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:28.895402 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ce16a-77b9-4c17-aae9-6cebbe0e8321 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:28.896874 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ce16a-77b9-4c17-aae9-6cebbe0e8321 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:28.913641 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48a169-e8da-43f1-a94b-81097bd3ba8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:28.914385 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48a169-e8da-43f1-a94b-81097bd3ba8f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:28.914663 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48a169-e8da-43f1-a94b-81097bd3ba8f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:28.914938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48a169-e8da-43f1-a94b-81097bd3ba8f None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:28.915656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c48a169-e8da-43f1-a94b-81097bd3ba8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:28.915958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c48a169-e8da-43f1-a94b-81097bd3ba8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:28.924678 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c48a169-e8da-43f1-a94b-81097bd3ba8f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:28.933449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-9d5ce16a-77b9-4c17-aae9-6cebbe0e8321 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:28.933449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:28.933449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:28.933449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:28.933449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:28.933449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:28.933449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:28.933449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:28.933449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:28.933449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:28.933449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:28.933449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:28.933449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:28.933449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:28.933449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:28.936148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:28.936148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:28.936148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:28.936148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:28.936148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:28.936148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:28.936148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:28.936148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:28.936148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:28.936148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:28.936148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:28.936148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:28.936148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:28.936148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:28.936148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:28.936148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:28.936148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:28.938286 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:28.938286 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:28.938286 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:28.938286 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:28.938286 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 362/729] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:28.941791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ce16a-77b9-4c17-aae9-6cebbe0e8321 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:28.942101 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 368/730] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:28.946898 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48a169-e8da-43f1-a94b-81097bd3ba8f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:29.007604 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:29.020080 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:29.062785 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48a169-e8da-43f1-a94b-81097bd3ba8f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:29.083686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ce16a-77b9-4c17-aae9-6cebbe0e8321 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:29.103333 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd8600-ddb4-4941-9cda-81388c780bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:29.104159 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd8600-ddb4-4941-9cda-81388c780bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:29.104444 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd8600-ddb4-4941-9cda-81388c780bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:29.104720 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd8600-ddb4-4941-9cda-81388c780bd8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:29.113373 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd8600-ddb4-4941-9cda-81388c780bd8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:29.114817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd8600-ddb4-4941-9cda-81388c780bd8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:29.127442 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642754d9-2148-4e5a-990a-59748e29fbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:29.128362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642754d9-2148-4e5a-990a-59748e29fbb5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:29.128839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642754d9-2148-4e5a-990a-59748e29fbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:29.129368 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642754d9-2148-4e5a-990a-59748e29fbb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:29.130454 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642754d9-2148-4e5a-990a-59748e29fbb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:29.130756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-dbdd8600-ddb4-4941-9cda-81388c780bd8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:20:29.130756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:29.130756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:29.130756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:29.130756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:29.130756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:29.130756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:29.130756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:29.130756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:29.130756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:29.130756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:29.130756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:29.130756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:29.130756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:29.130756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:29.130756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:29.131656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:29.131656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:29.131656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:29.131656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:29.131656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:29.131656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:29.131656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:29.131656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:29.131656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:29.131656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:29.131656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:29.131656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:29.131656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:29.131656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:29.131656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:29.131656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:29.131656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:29.131656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:29.132584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:29.132584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:29.132584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:29.132584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:29.132584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:29.132584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:29.132584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:29.132584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:29.132584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:20:29.132584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:29.133877 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642754d9-2148-4e5a-990a-59748e29fbb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:29.135469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642754d9-2148-4e5a-990a-59748e29fbb5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:29.136634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 369/731] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jun 7 08:20:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:29.138666 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e02d78-15e6-4b1a-8dda-405d6f736396 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:29.138958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e02d78-15e6-4b1a-8dda-405d6f736396 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:29.139715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e02d78-15e6-4b1a-8dda-405d6f736396 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:29.144990 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 370/732] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:29.145968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e02d78-15e6-4b1a-8dda-405d6f736396 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:29.150395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 363/733] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:29.155472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0601014-1c0b-41d8-a4f6-c95a736a3b24 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:29.156168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0601014-1c0b-41d8-a4f6-c95a736a3b24 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:29.156652 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0601014-1c0b-41d8-a4f6-c95a736a3b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:29.174256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a0601014-1c0b-41d8-a4f6-c95a736a3b24 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:29.175818 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a0601014-1c0b-41d8-a4f6-c95a736a3b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:29.245196 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 364/734] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:29.246373 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0601014-1c0b-41d8-a4f6-c95a736a3b24 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:29.274538 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e02d78-15e6-4b1a-8dda-405d6f736396 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:29.278815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:29.307421 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:29.319258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5987c-ba1a-41fd-be0c-4889c8981b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:29.319930 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5987c-ba1a-41fd-be0c-4889c8981b53 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:29.320210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5987c-ba1a-41fd-be0c-4889c8981b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:29.320487 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5987c-ba1a-41fd-be0c-4889c8981b53 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:29.321384 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a5987c-ba1a-41fd-be0c-4889c8981b53 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:29.321702 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a5987c-ba1a-41fd-be0c-4889c8981b53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:29.323211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a5987c-ba1a-41fd-be0c-4889c8981b53 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:29.341398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 371/735] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Jun 7 08:20:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:29.346014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21df47-3e26-4f42-b045-a73371999ae4 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:29.346014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21df47-3e26-4f42-b045-a73371999ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:29.346014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21df47-3e26-4f42-b045-a73371999ae4 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:29.346014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 372/736] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:29.346014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21df47-3e26-4f42-b045-a73371999ae4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:29.364861 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0601014-1c0b-41d8-a4f6-c95a736a3b24 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:29.399856 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c704fbd-3481-435c-ab77-bdc54080e350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:29.400637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c704fbd-3481-435c-ab77-bdc54080e350 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:29.400921 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c704fbd-3481-435c-ab77-bdc54080e350 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:29.401213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c704fbd-3481-435c-ab77-bdc54080e350 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:29.408386 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c704fbd-3481-435c-ab77-bdc54080e350 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:29.409872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c704fbd-3481-435c-ab77-bdc54080e350 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:29.424796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-2c704fbd-3481-435c-ab77-bdc54080e350 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:20:29.424796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:29.424796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:29.424796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:29.424796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:29.424796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:29.424796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:29.424796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:29.424796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:29.424796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:29.424796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:29.424796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:29.424796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:29.424796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:29.424796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:29.424796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:29.425815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:29.425815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:29.425815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:29.425815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:29.425815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:29.425815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:29.425815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:29.425815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:29.425815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:29.425815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:29.425815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:29.425815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:29.425815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:29.425815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:29.425815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:29.425815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:29.425815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:29.425815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:29.426786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:29.426786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:29.426786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:29.426786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:29.426786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:29.426786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:29.426786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:29.426786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:29.426786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:20:29.426786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:29.426786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 365/737] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Jun 7 08:20:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:29.427398 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53095a-998a-4c7a-ba46-5aeae7f9557e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:29.427469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53095a-998a-4c7a-ba46-5aeae7f9557e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:29.427879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53095a-998a-4c7a-ba46-5aeae7f9557e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:29.445654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7e53095a-998a-4c7a-ba46-5aeae7f9557e None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:29.446838 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7e53095a-998a-4c7a-ba46-5aeae7f9557e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:29.460275 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21df47-3e26-4f42-b045-a73371999ae4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:29.495082 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640e4f8-61ad-4abd-9c06-e64d2107a633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:29.495749 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640e4f8-61ad-4abd-9c06-e64d2107a633 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:29.496031 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640e4f8-61ad-4abd-9c06-e64d2107a633 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:29.496303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640e4f8-61ad-4abd-9c06-e64d2107a633 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:29.497066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a640e4f8-61ad-4abd-9c06-e64d2107a633 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:29.499043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a640e4f8-61ad-4abd-9c06-e64d2107a633 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:29.518277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 366/738] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:29.519167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53095a-998a-4c7a-ba46-5aeae7f9557e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:29.547449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:29.613508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 373/739] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jun 7 08:20:29 2022] POST /identity/v3/services => generated 240 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:29.614504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640e4f8-61ad-4abd-9c06-e64d2107a633 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:29.627663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53095a-998a-4c7a-ba46-5aeae7f9557e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:29.678823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:29.688486 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ecf490-995b-40e2-9ecc-8c7ce9aa8442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:29.689600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ecf490-995b-40e2-9ecc-8c7ce9aa8442 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:29.692270 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ecf490-995b-40e2-9ecc-8c7ce9aa8442 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:29.692270 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ecf490-995b-40e2-9ecc-8c7ce9aa8442 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:29.759521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640e4f8-61ad-4abd-9c06-e64d2107a633 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:29.776666 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ecf490-995b-40e2-9ecc-8c7ce9aa8442 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:29.778491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ecf490-995b-40e2-9ecc-8c7ce9aa8442 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:29.799091 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd01d7-a842-4084-9a25-b2e920c14383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:29.799998 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd01d7-a842-4084-9a25-b2e920c14383 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:29.800111 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd01d7-a842-4084-9a25-b2e920c14383 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:29.800312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd01d7-a842-4084-9a25-b2e920c14383 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:29.801207 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd01d7-a842-4084-9a25-b2e920c14383 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:29.801515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd01d7-a842-4084-9a25-b2e920c14383 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:29.802951 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd01d7-a842-4084-9a25-b2e920c14383 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:29.803805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:29.824223 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 374/740] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Jun 7 08:20:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:29.825642 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd01d7-a842-4084-9a25-b2e920c14383 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:29.896644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 367/741] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:29 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 08:20:29.898287 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ecf490-995b-40e2-9ecc-8c7ce9aa8442 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:29.938837 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:29.942911 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd01d7-a842-4084-9a25-b2e920c14383 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:29.977714 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc5031-d3fe-4813-a32c-5e66a2f28e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:29.978672 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc5031-d3fe-4813-a32c-5e66a2f28e53 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:29.978987 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc5031-d3fe-4813-a32c-5e66a2f28e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:29.979276 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc5031-d3fe-4813-a32c-5e66a2f28e53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:29.992547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc5031-d3fe-4813-a32c-5e66a2f28e53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:30.000332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-0adc5031-d3fe-4813-a32c-5e66a2f28e53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:30.000332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:30.000332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:30.000332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:30.000332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:30.000332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:30.000332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:30.000332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:30.000332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:30.000332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:30.000332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:30.000332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:30.000332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:30.000332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:30.000332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:30.000332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:30.001396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:30.001396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:30.001396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:30.001396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:30.001396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:30.001396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:30.001396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:30.001396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:30.001396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:30.001396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:30.001396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:30.001396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:30.001396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:30.001396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:30.001396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:30.001396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:30.002210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:30.002210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:30.002210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:30.002210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:30.002210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:30.002210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:30.002210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:30.002210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:30.002210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:30.002210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:30.002210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:30.002210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:30.003453 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 375/742] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:20:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:30.007423 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ecf490-995b-40e2-9ecc-8c7ce9aa8442 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:30.008952 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929205db-d4f4-4f55-ac01-071517871e61 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:30.009261 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929205db-d4f4-4f55-ac01-071517871e61 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:30.009539 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929205db-d4f4-4f55-ac01-071517871e61 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:30.010975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 376/743] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:30.011529 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929205db-d4f4-4f55-ac01-071517871e61 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:30.029241 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:30.046858 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25645892-7a8f-4cea-95b2-3a0200ee7d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:30.047661 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25645892-7a8f-4cea-95b2-3a0200ee7d38 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:30.047952 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25645892-7a8f-4cea-95b2-3a0200ee7d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:30.048227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25645892-7a8f-4cea-95b2-3a0200ee7d38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:30.059384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25645892-7a8f-4cea-95b2-3a0200ee7d38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:30.060866 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25645892-7a8f-4cea-95b2-3a0200ee7d38 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:30.078954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-25645892-7a8f-4cea-95b2-3a0200ee7d38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:30.078954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:30.078954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:30.078954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:30.078954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:30.078954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:30.078954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:30.078954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:30.078954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:30.078954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:30.078954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:30.078954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:30.078954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:30.078954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:30.078954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:30.080026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:30.080026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:30.080026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:30.080026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:30.080026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:30.080026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:30.080026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:30.080026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:30.080026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:30.080026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:30.080026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:30.080026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:30.080026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:30.080026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:30.080026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:30.080026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:30.080026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:30.080916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:30.080916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:30.080916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:30.080916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:30.080916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 368/744] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:30.080916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25645892-7a8f-4cea-95b2-3a0200ee7d38 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:30.125129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929205db-d4f4-4f55-ac01-071517871e61 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:30.162808 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f848e-d2bb-46ab-b6f8-cbf011000cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:30.163644 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f848e-d2bb-46ab-b6f8-cbf011000cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:30.163928 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f848e-d2bb-46ab-b6f8-cbf011000cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:30.164208 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f848e-d2bb-46ab-b6f8-cbf011000cc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:30.175066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f848e-d2bb-46ab-b6f8-cbf011000cc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:30.176171 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f848e-d2bb-46ab-b6f8-cbf011000cc4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:30.194045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-aa1f848e-d2bb-46ab-b6f8-cbf011000cc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:30.194045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:30.194045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:30.194045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:30.194045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:30.194045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:30.194045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:30.194045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:30.194045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:30.194045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:30.194045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:30.194045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:30.194045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:30.194045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:30.194045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:30.194915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:30.194915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:30.194915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:30.194915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:30.194915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:30.194915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:30.194915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:30.194915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:30.194915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:30.194915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:30.194915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:30.194915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:30.194915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:30.194915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:30.194915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:30.194915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:30.194915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:30.195813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:30.195813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:30.195813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:30.195813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:30.195813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 377/745] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:30.196691 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4769d9f-5a7e-48e8-9f30-2461924cbce0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:30.196793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4769d9f-5a7e-48e8-9f30-2461924cbce0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:30.197053 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4769d9f-5a7e-48e8-9f30-2461924cbce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:30.207661 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25645892-7a8f-4cea-95b2-3a0200ee7d38 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:30.213725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c4769d9f-5a7e-48e8-9f30-2461924cbce0 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:30.214931 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c4769d9f-5a7e-48e8-9f30-2461924cbce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:30.219463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:30.245186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794e2fe-574c-4e10-ab4c-560bdce8a843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:30.246027 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794e2fe-574c-4e10-ab4c-560bdce8a843 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:30.246659 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794e2fe-574c-4e10-ab4c-560bdce8a843 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:30.246659 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794e2fe-574c-4e10-ab4c-560bdce8a843 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:30.247629 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b794e2fe-574c-4e10-ab4c-560bdce8a843 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:30.248030 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b794e2fe-574c-4e10-ab4c-560bdce8a843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:30.249541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b794e2fe-574c-4e10-ab4c-560bdce8a843 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:30.257997 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 369/746] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:30.258706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794e2fe-574c-4e10-ab4c-560bdce8a843 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:30.291019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:30.293075 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 378/747] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:30 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 08:20:30.293914 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4769d9f-5a7e-48e8-9f30-2461924cbce0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:30.368403 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794e2fe-574c-4e10-ab4c-560bdce8a843 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:30.405793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b2222-ec82-4490-a24c-990bb4bb0d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:30.406451 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b2222-ec82-4490-a24c-990bb4bb0d08 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:30.406733 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b2222-ec82-4490-a24c-990bb4bb0d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:30.407015 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b2222-ec82-4490-a24c-990bb4bb0d08 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:30.407780 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b2222-ec82-4490-a24c-990bb4bb0d08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:30.408083 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b2222-ec82-4490-a24c-990bb4bb0d08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:30.409495 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b2222-ec82-4490-a24c-990bb4bb0d08 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:30.413127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4769d9f-5a7e-48e8-9f30-2461924cbce0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:30.421251 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 370/748] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:30.422122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b2222-ec82-4490-a24c-990bb4bb0d08 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:30.454461 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c21a5-4756-4f28-85bd-3f8dd0d98d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:30.455437 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c21a5-4756-4f28-85bd-3f8dd0d98d63 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:30.455891 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c21a5-4756-4f28-85bd-3f8dd0d98d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:30.456337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c21a5-4756-4f28-85bd-3f8dd0d98d63 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:30.457312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c21a5-4756-4f28-85bd-3f8dd0d98d63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:30.457803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c21a5-4756-4f28-85bd-3f8dd0d98d63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:30.459181 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c21a5-4756-4f28-85bd-3f8dd0d98d63 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:30.468153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:30.476680 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 379/749] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:30.477517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c21a5-4756-4f28-85bd-3f8dd0d98d63 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:30.518085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:30.528099 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b2222-ec82-4490-a24c-990bb4bb0d08 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:30.565842 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8cef0-d09f-4b40-8aa1-601470fcc528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:30.566782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8cef0-d09f-4b40-8aa1-601470fcc528 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:30.567075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8cef0-d09f-4b40-8aa1-601470fcc528 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:30.567351 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8cef0-d09f-4b40-8aa1-601470fcc528 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:30.578697 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8cef0-d09f-4b40-8aa1-601470fcc528 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:30.580140 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8cef0-d09f-4b40-8aa1-601470fcc528 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:30.583686 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c21a5-4756-4f28-85bd-3f8dd0d98d63 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:30.597340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-20c8cef0-d09f-4b40-8aa1-601470fcc528 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:30.597340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:30.597340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:30.597340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:30.597340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:30.597340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:30.597340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:30.597340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:30.597340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:30.597340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:30.597340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:30.597340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:30.597340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:30.597340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:30.597340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:30.598219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:30.598219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:30.598219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:30.598219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:30.598219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:30.598219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:30.598219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:30.598219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:30.598219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:30.598219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:30.598219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:30.598219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:30.598219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:30.598219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:30.598219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:30.598219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:30.598219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:30.599102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:30.599102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:30.599102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:30.599102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:30.599102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 371/750] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:30.599383 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8cef0-d09f-4b40-8aa1-601470fcc528 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:30.625172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeeed08-6dcd-46ad-ba9c-de641e88ac45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:30.625838 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeed08-6dcd-46ad-ba9c-de641e88ac45 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:30.626111 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeed08-6dcd-46ad-ba9c-de641e88ac45 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:30.626381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeed08-6dcd-46ad-ba9c-de641e88ac45 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:30.627090 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeeed08-6dcd-46ad-ba9c-de641e88ac45 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:30.628438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeeed08-6dcd-46ad-ba9c-de641e88ac45 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:30.676969 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 380/751] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jun 7 08:20:30 2022] POST /identity/v3/services => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:30.678581 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeeed08-6dcd-46ad-ba9c-de641e88ac45 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:30.705789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8cef0-d09f-4b40-8aa1-601470fcc528 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:30.751628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73955c13-9e65-4816-95b0-3f5f00d9c6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:30.752440 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73955c13-9e65-4816-95b0-3f5f00d9c6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:30.752716 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73955c13-9e65-4816-95b0-3f5f00d9c6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:30.752987 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73955c13-9e65-4816-95b0-3f5f00d9c6a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:30.761446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73955c13-9e65-4816-95b0-3f5f00d9c6a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:30.763086 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73955c13-9e65-4816-95b0-3f5f00d9c6a6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:30.767258 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:30.772746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:30.779994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-73955c13-9e65-4816-95b0-3f5f00d9c6a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:30.779994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:30.779994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:30.779994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:30.779994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:30.779994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:30.779994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:30.779994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:30.779994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:30.779994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:30.779994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:30.779994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:30.779994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:30.779994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:30.779994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:30.780972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:30.780972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:30.780972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:30.780972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:30.780972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:30.780972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:30.780972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:30.780972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:30.780972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:30.780972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:30.780972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:30.780972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:30.780972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:30.780972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:30.780972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:30.780972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:30.780972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:30.781896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:30.781896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:30.781896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:30.781896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:30.781896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:30.781896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:30.781896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:30.781896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:30.781896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 372/752] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:30.782364 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73955c13-9e65-4816-95b0-3f5f00d9c6a6 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:30.830688 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeeed08-6dcd-46ad-ba9c-de641e88ac45 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:30.873372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbfe95c-ebb8-44ab-9c20-b3102a93786e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:30.874187 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbfe95c-ebb8-44ab-9c20-b3102a93786e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:30.874457 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbfe95c-ebb8-44ab-9c20-b3102a93786e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:30.874726 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbfe95c-ebb8-44ab-9c20-b3102a93786e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:30.882321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbfe95c-ebb8-44ab-9c20-b3102a93786e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:30.883969 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbfe95c-ebb8-44ab-9c20-b3102a93786e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:30.899392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-6dbfe95c-ebb8-44ab-9c20-b3102a93786e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:30.899392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:30.899392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:30.899392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:30.899392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:30.899392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:30.899392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:30.899392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:30.899392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:30.899392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:30.899392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:30.899392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:30.899392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:30.899392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:30.899392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:30.900368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:30.900368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:30.900368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:30.900368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:30.900368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:30.900368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:30.900368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:30.900368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:30.900368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:30.900368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:30.900368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:30.900368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:30.900368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:30.900368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:30.900368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:30.900368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:30.900368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:30.901290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:30.901290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:30.901290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:30.901290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:30.901290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:30.901290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:30.901290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:30.901290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:30.906008 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 381/753] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:30.906846 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbfe95c-ebb8-44ab-9c20-b3102a93786e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:30.916289 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73955c13-9e65-4816-95b0-3f5f00d9c6a6 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:30.959688 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928410d-194d-4ce2-9e4c-9c5517af3213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:30.960282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928410d-194d-4ce2-9e4c-9c5517af3213 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:30.960550 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928410d-194d-4ce2-9e4c-9c5517af3213 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:30.960822 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928410d-194d-4ce2-9e4c-9c5517af3213 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:30.962024 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928410d-194d-4ce2-9e4c-9c5517af3213 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:30.962024 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928410d-194d-4ce2-9e4c-9c5517af3213 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:30.963257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928410d-194d-4ce2-9e4c-9c5517af3213 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:30.981085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 373/754] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:30.982498 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928410d-194d-4ce2-9e4c-9c5517af3213 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:31.028378 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbfe95c-ebb8-44ab-9c20-b3102a93786e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:31.042370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:31.051399 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:31.066006 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41b6b5-42e7-4c1f-9885-48f746583594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:31.066954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41b6b5-42e7-4c1f-9885-48f746583594 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:31.067473 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41b6b5-42e7-4c1f-9885-48f746583594 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:31.068021 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41b6b5-42e7-4c1f-9885-48f746583594 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:31.069049 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41b6b5-42e7-4c1f-9885-48f746583594 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:31.069563 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41b6b5-42e7-4c1f-9885-48f746583594 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:31.071227 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41b6b5-42e7-4c1f-9885-48f746583594 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:31.088270 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928410d-194d-4ce2-9e4c-9c5517af3213 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:31.088680 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 382/755] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:31.089901 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41b6b5-42e7-4c1f-9885-48f746583594 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:31.120022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cac673-9d18-4c86-b017-fbb5c2a5fcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:31.120881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cac673-9d18-4c86-b017-fbb5c2a5fcad None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:31.121257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cac673-9d18-4c86-b017-fbb5c2a5fcad None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:31.121469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cac673-9d18-4c86-b017-fbb5c2a5fcad None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:31.122235 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cac673-9d18-4c86-b017-fbb5c2a5fcad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:31.122565 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cac673-9d18-4c86-b017-fbb5c2a5fcad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:31.123987 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cac673-9d18-4c86-b017-fbb5c2a5fcad None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:31.135827 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 374/756] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:31.136615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cac673-9d18-4c86-b017-fbb5c2a5fcad None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:31.213169 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41b6b5-42e7-4c1f-9885-48f746583594 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:31.248285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ba6e7-58af-4ed0-bb1c-e548841643fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:31.249114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ba6e7-58af-4ed0-bb1c-e548841643fd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:31.249395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ba6e7-58af-4ed0-bb1c-e548841643fd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:31.249668 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ba6e7-58af-4ed0-bb1c-e548841643fd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:31.257699 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ba6e7-58af-4ed0-bb1c-e548841643fd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:31.259286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ba6e7-58af-4ed0-bb1c-e548841643fd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:31.271032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cac673-9d18-4c86-b017-fbb5c2a5fcad None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:31.274553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-c56ba6e7-58af-4ed0-bb1c-e548841643fd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:20:31.274553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:31.274553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:31.274553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:31.274553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:31.274553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:31.274553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:31.274553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:31.274553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:31.274553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:31.274553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:31.274553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:31.274553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:31.274553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:31.274553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:31.274553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:31.275482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:31.275482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:31.275482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:31.275482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:31.275482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:31.275482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:31.275482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:31.275482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:31.275482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:31.275482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:31.275482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:31.275482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:31.275482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:31.275482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:31.275482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:31.275482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:31.275482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:31.275482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:31.276415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:31.276415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:31.276415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:31.276415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:31.276415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:31.276415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:31.276415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:31.276415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:31.276415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:20:31.276415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:31.280557 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 383/757] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:20:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:31.281494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ba6e7-58af-4ed0-bb1c-e548841643fd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:31.299338 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:31.319602 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7ca5c-9c73-4178-9392-f001620c3ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:31.321089 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7ca5c-9c73-4178-9392-f001620c3ded None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:31.321583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7ca5c-9c73-4178-9392-f001620c3ded None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:31.322859 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7ca5c-9c73-4178-9392-f001620c3ded None admin] PATH_INFO: `/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/ee42568b059b49a198ed2c01d5efdafd/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:31.325132 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7ca5c-9c73-4178-9392-f001620c3ded None admin] RBAC: Authorizing `identity:create_grant(project_id=afc8781e5f864d4a9374300ee9b5193b, user_id=ee42568b059b49a198ed2c01d5efdafd, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:31.328462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7ca5c-9c73-4178-9392-f001620c3ded None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:31.330167 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:31.392667 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ba6e7-58af-4ed0-bb1c-e548841643fd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:31.405454 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 375/758] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:20:31 2022] PUT /identity/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/ee42568b059b49a198ed2c01d5efdafd/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:20:31.406901 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7ca5c-9c73-4178-9392-f001620c3ded None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:31.441760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e745d-b189-4800-bc8b-9f6fe606f9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:31.442666 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e745d-b189-4800-bc8b-9f6fe606f9fa None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:31.442781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e745d-b189-4800-bc8b-9f6fe606f9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:31.443004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e745d-b189-4800-bc8b-9f6fe606f9fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:31.443838 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e745d-b189-4800-bc8b-9f6fe606f9fa None admin] RBAC: Adding query filter params (role.id=a04f2ab5f1234bd6b1dc4c18b4210a89, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=bbf0ce396885472bb5d225aa009043f2) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:31.444159 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e745d-b189-4800-bc8b-9f6fe606f9fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:31.445798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e745d-b189-4800-bc8b-9f6fe606f9fa None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:31.458875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 384/759] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:31 2022] GET /identity/v3/role_assignments?user.id=bbf0ce396885472bb5d225aa009043f2&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:31.532477 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7ca5c-9c73-4178-9392-f001620c3ded None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:31.574453 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f026a-4244-40c3-91a4-7833ece3f44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:31.576155 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f026a-4244-40c3-91a4-7833ece3f44e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:31.576655 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f026a-4244-40c3-91a4-7833ece3f44e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:31.577185 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f026a-4244-40c3-91a4-7833ece3f44e None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:31.578330 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f026a-4244-40c3-91a4-7833ece3f44e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:31.578923 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f026a-4244-40c3-91a4-7833ece3f44e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:31.580675 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f026a-4244-40c3-91a4-7833ece3f44e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:31.603562 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 376/760] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Jun 7 08:20:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:31.609680 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e745d-b189-4800-bc8b-9f6fe606f9fa None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:31.748737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e745d-b189-4800-bc8b-9f6fe606f9fa None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:31.751499 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:31.788811 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd98086-a25a-4827-a4fa-08f897600a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:31.789677 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd98086-a25a-4827-a4fa-08f897600a0d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:31.790140 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd98086-a25a-4827-a4fa-08f897600a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:31.790711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd98086-a25a-4827-a4fa-08f897600a0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:31.864879 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd98086-a25a-4827-a4fa-08f897600a0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:31.866366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd98086-a25a-4827-a4fa-08f897600a0d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:31.937616 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce3901-1cab-46a1-97f9-69b1d9b682a8 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:31.938479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce3901-1cab-46a1-97f9-69b1d9b682a8 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:31.938761 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce3901-1cab-46a1-97f9-69b1d9b682a8 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:31.940254 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 377/761] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:31.945576 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8ee84-5cda-4803-813a-29c06f6f89ab None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:31.946642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8ee84-5cda-4803-813a-29c06f6f89ab None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:31.947127 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8ee84-5cda-4803-813a-29c06f6f89ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:31.966859 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a3b8ee84-5cda-4803-813a-29c06f6f89ab None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:31.968848 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a3b8ee84-5cda-4803-813a-29c06f6f89ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:31.988725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 385/762] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:20:31.999852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd98086-a25a-4827-a4fa-08f897600a0d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:32.005357 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:32.030605 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:32.108767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 378/763] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:20:32.134783 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8ee84-5cda-4803-813a-29c06f6f89ab None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:32.186156 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd98086-a25a-4827-a4fa-08f897600a0d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:32.261735 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8ee84-5cda-4803-813a-29c06f6f89ab None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:32.271148 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a292b7-bb75-4bc5-aa67-35c5b9cbd777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:32.271967 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a292b7-bb75-4bc5-aa67-35c5b9cbd777 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:32.272251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a292b7-bb75-4bc5-aa67-35c5b9cbd777 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:32.272525 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a292b7-bb75-4bc5-aa67-35c5b9cbd777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:32.302336 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a292b7-bb75-4bc5-aa67-35c5b9cbd777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:32.303801 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a292b7-bb75-4bc5-aa67-35c5b9cbd777 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:32.304853 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa372f63-1c29-4b98-9c68-dfcf59277824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:32.305706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa372f63-1c29-4b98-9c68-dfcf59277824 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:32.305984 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa372f63-1c29-4b98-9c68-dfcf59277824 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:32.306254 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa372f63-1c29-4b98-9c68-dfcf59277824 None admin] PATH_INFO: `/v3/services/network` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:32.306906 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa372f63-1c29-4b98-9c68-dfcf59277824 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:32.308265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa372f63-1c29-4b98-9c68-dfcf59277824 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:32.327710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-aa372f63-1c29-4b98-9c68-dfcf59277824 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 08:20:32.327710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:32.327710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:20:32.327710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:20:32.327710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:20:32.327710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:20:32.327710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:20:32.327710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:32.327710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 08:20:32.327710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:32.327710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:32.327710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:32.327710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:32.327710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:32.327710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:32.327710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:32.327710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:32.328710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:32.328710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:32.328710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:32.328710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:32.328710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:32.328710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:32.328710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:20:32.328710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:20:32.328710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:20:32.328710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:20:32.328710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:32.328710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:32.328710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:32.328710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:32.328710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:32.328710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:32.329575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:32.329575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:32.329575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:32.329575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:32.329575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:32.329575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:32.329575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:32.329575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:32.329575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:32.329575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:32.329575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:20:32.329575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:32.329575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 08:20:32.329575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:32.329575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 379/764] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:32 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:32.334158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-52a292b7-bb75-4bc5-aa67-35c5b9cbd777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:32.334158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:32.334158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:32.334158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:32.334158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:32.334158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:32.334158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:32.334158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:32.334158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:32.334158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:32.334158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:32.334158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:32.334158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:32.334158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:32.334158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:32.335002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:32.335002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:32.335002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:32.335002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:32.335002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:32.335002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:32.335002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:32.335002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:32.335002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:32.335002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:32.335002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:32.335002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:32.335002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:32.335002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:32.335002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:32.335002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:32.335002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:32.335847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:32.335847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:32.335847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:32.335847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:32.335847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 386/765] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:32.340764 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa372f63-1c29-4b98-9c68-dfcf59277824 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:32.340961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a292b7-bb75-4bc5-aa67-35c5b9cbd777 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:32.374438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:32.446437 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa372f63-1c29-4b98-9c68-dfcf59277824 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:32.465347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:32.496273 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787ba88-4632-4cd8-b68e-3ea646040184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:32.496955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787ba88-4632-4cd8-b68e-3ea646040184 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:32.497297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787ba88-4632-4cd8-b68e-3ea646040184 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:32.497578 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787ba88-4632-4cd8-b68e-3ea646040184 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:32.498295 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787ba88-4632-4cd8-b68e-3ea646040184 None admin] RBAC: Adding query filter params (name=network) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:32.498604 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787ba88-4632-4cd8-b68e-3ea646040184 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:32.501211 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787ba88-4632-4cd8-b68e-3ea646040184 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:32.510163 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 380/766] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jun 7 08:20:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:32.516326 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787ba88-4632-4cd8-b68e-3ea646040184 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:32.547067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a292b7-bb75-4bc5-aa67-35c5b9cbd777 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:32.611098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ceef2-ceeb-4712-985f-963c26568f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:32.611816 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ceef2-ceeb-4712-985f-963c26568f8c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:32.612098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ceef2-ceeb-4712-985f-963c26568f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:32.612371 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ceef2-ceeb-4712-985f-963c26568f8c None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:32.613112 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ceef2-ceeb-4712-985f-963c26568f8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:32.613419 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ceef2-ceeb-4712-985f-963c26568f8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:32.614792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ceef2-ceeb-4712-985f-963c26568f8c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:32.647151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 387/767] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:32.657972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ceef2-ceeb-4712-985f-963c26568f8c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:32.665460 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:32.671326 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787ba88-4632-4cd8-b68e-3ea646040184 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:32.710211 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175586b8-2a20-4b9c-89ba-30df8e3c4c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:32.711155 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175586b8-2a20-4b9c-89ba-30df8e3c4c8f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:32.711617 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175586b8-2a20-4b9c-89ba-30df8e3c4c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:32.712170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175586b8-2a20-4b9c-89ba-30df8e3c4c8f None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:32.713076 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175586b8-2a20-4b9c-89ba-30df8e3c4c8f None admin] RBAC: Adding query filter params (type=network) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:32.713562 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175586b8-2a20-4b9c-89ba-30df8e3c4c8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:32.715485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175586b8-2a20-4b9c-89ba-30df8e3c4c8f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:32.722539 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 381/768] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jun 7 08:20:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:32.733017 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175586b8-2a20-4b9c-89ba-30df8e3c4c8f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:32.844431 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ceef2-ceeb-4712-985f-963c26568f8c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:32.876341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:32.907289 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175586b8-2a20-4b9c-89ba-30df8e3c4c8f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:32.910262 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afcfd8-bca4-4b8b-9ae6-5955159029cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:32.911304 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afcfd8-bca4-4b8b-9ae6-5955159029cc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:32.911761 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afcfd8-bca4-4b8b-9ae6-5955159029cc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:32.912257 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afcfd8-bca4-4b8b-9ae6-5955159029cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:32.930616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4afcfd8-bca4-4b8b-9ae6-5955159029cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:32.932586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4afcfd8-bca4-4b8b-9ae6-5955159029cc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:32.943079 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f0cad-ff1f-4bbd-9624-ff818b823fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:32.943742 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f0cad-ff1f-4bbd-9624-ff818b823fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:32.944005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f0cad-ff1f-4bbd-9624-ff818b823fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:32.944278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f0cad-ff1f-4bbd-9624-ff818b823fa6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:32.945166 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f0cad-ff1f-4bbd-9624-ff818b823fa6 None admin] RBAC: Adding query filter params (interface=public, service_id=7b72e4bbe2a24f7fa4eb6b36008f7914, region_id=RegionOne) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:32.945476 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f0cad-ff1f-4bbd-9624-ff818b823fa6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:32.946938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f0cad-ff1f-4bbd-9624-ff818b823fa6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:32.951067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-b4afcfd8-bca4-4b8b-9ae6-5955159029cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:32.951067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:32.951067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:32.951067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:32.951067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:32.951067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:32.951067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:32.951067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:32.951067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:32.951067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:32.951067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:32.951067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:32.951067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:32.951067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:32.951067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:32.951933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:32.951933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:32.951933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:32.951933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:32.951933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:32.951933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:32.951933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:32.951933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:32.951933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:32.951933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:32.951933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:32.951933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:32.951933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:32.951933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:32.951933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:32.951933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:32.951933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:32.952815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:32.952815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:32.952815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:32.952815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:32.952815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:32.952815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:32.952815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:32.952815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:32.959903 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 382/770] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Jun 7 08:20:32 2022] GET /identity/v3/endpoints?service_id=7b72e4bbe2a24f7fa4eb6b36008f7914&interface=public®ion_id=RegionOne => generated 194 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:32.960613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 388/770] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:32.961772 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f0cad-ff1f-4bbd-9624-ff818b823fa6 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:32.962207 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afcfd8-bca4-4b8b-9ae6-5955159029cc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:33.022063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:33.123949 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f0cad-ff1f-4bbd-9624-ff818b823fa6 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:33.134958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afcfd8-bca4-4b8b-9ae6-5955159029cc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:33.166294 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bf0b0-c9a0-4736-8076-18d1c5ade71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:33.166996 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bf0b0-c9a0-4736-8076-18d1c5ade71f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:33.167281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bf0b0-c9a0-4736-8076-18d1c5ade71f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:33.167561 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bf0b0-c9a0-4736-8076-18d1c5ade71f None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:33.168244 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267bf0b0-c9a0-4736-8076-18d1c5ade71f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:33.170324 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267bf0b0-c9a0-4736-8076-18d1c5ade71f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:33.176640 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 383/771] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Jun 7 08:20:32 2022] GET /identity/v3/services => generated 772 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:33.206792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f210130-25aa-477c-bfe2-e6787392378c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:33.207488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f210130-25aa-477c-bfe2-e6787392378c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:33.207773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f210130-25aa-477c-bfe2-e6787392378c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:33.208046 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f210130-25aa-477c-bfe2-e6787392378c None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:33.208855 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f210130-25aa-477c-bfe2-e6787392378c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:33.209228 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f210130-25aa-477c-bfe2-e6787392378c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:33.210712 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f210130-25aa-477c-bfe2-e6787392378c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:33.230471 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 389/772] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:33.237778 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bf0b0-c9a0-4736-8076-18d1c5ade71f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:33.381371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:33.393081 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bf0b0-c9a0-4736-8076-18d1c5ade71f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:33.439686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7583e9-cf1e-4b03-b07e-617ed7606481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:33.440270 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7583e9-cf1e-4b03-b07e-617ed7606481 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:33.440551 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7583e9-cf1e-4b03-b07e-617ed7606481 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:33.440825 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7583e9-cf1e-4b03-b07e-617ed7606481 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:33.441694 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7583e9-cf1e-4b03-b07e-617ed7606481 None admin] RBAC: Adding query filter params (role.id=8c63c00418a346b29f7db3f87a404e70, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=334dc47ae2e643ccb44d25c6b5906f14) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:33.442007 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7583e9-cf1e-4b03-b07e-617ed7606481 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:33.443471 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7583e9-cf1e-4b03-b07e-617ed7606481 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:33.449956 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 384/773] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:33 2022] GET /identity/v3/role_assignments?user.id=334dc47ae2e643ccb44d25c6b5906f14&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:33.450671 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eed5e4-2c9f-4384-8168-e517041fd63d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:33.450963 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eed5e4-2c9f-4384-8168-e517041fd63d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:33.451243 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eed5e4-2c9f-4384-8168-e517041fd63d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:33.452880 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 390/774] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:33.458839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254c205-202a-4338-b589-1341d86fee4d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:33.459193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254c205-202a-4338-b589-1341d86fee4d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:33.459395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254c205-202a-4338-b589-1341d86fee4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:33.478290 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1254c205-202a-4338-b589-1341d86fee4d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:33.479594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1254c205-202a-4338-b589-1341d86fee4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:33.559073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 385/775] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:33.583446 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eed5e4-2c9f-4384-8168-e517041fd63d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:33.633958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:33.721134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eed5e4-2c9f-4384-8168-e517041fd63d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:33.762346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5c02c-460e-44d9-bb8c-e02847dadd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:33.763203 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5c02c-460e-44d9-bb8c-e02847dadd71 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:33.763486 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5c02c-460e-44d9-bb8c-e02847dadd71 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:33.763758 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5c02c-460e-44d9-bb8c-e02847dadd71 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:33.764411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5c02c-460e-44d9-bb8c-e02847dadd71 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:33.765815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5c02c-460e-44d9-bb8c-e02847dadd71 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:33.779929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-25b5c02c-460e-44d9-bb8c-e02847dadd71 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 08:20:33.779929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:33.779929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:20:33.779929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:20:33.779929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:20:33.779929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:20:33.779929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:20:33.779929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:33.779929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 08:20:33.779929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:33.779929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:33.779929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:33.779929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:33.779929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:33.779929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:33.779929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:33.780910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:33.780910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:33.780910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:33.780910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:33.780910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:33.780910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:33.780910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:33.780910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:20:33.780910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:20:33.780910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:20:33.780910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:20:33.780910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:33.780910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:33.780910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:33.780910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:33.780910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:33.782251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:33.782251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:33.782251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:33.782251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:33.782251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:33.782251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:33.782251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:33.782251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:33.782251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:33.782251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:33.782251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:33.782251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:20:33.782251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:33.782251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 08:20:33.782251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:33.782251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 391/776] 10.222.0.17 () {56 vars in 1129 bytes} [Tue Jun 7 08:20:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:33.790116 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254c205-202a-4338-b589-1341d86fee4d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:33.925424 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:33.951185 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254c205-202a-4338-b589-1341d86fee4d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:34.005894 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bc2d2-45b5-4c66-8572-39bc4c4fa07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:34.006513 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bc2d2-45b5-4c66-8572-39bc4c4fa07d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:34.006797 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bc2d2-45b5-4c66-8572-39bc4c4fa07d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:34.007077 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bc2d2-45b5-4c66-8572-39bc4c4fa07d None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:34.007955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bc2d2-45b5-4c66-8572-39bc4c4fa07d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:34.008269 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bc2d2-45b5-4c66-8572-39bc4c4fa07d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:34.009753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bc2d2-45b5-4c66-8572-39bc4c4fa07d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:34.015978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 386/777] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Jun 7 08:20:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:34.021556 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5c02c-460e-44d9-bb8c-e02847dadd71 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:34.129689 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:34.135759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5c02c-460e-44d9-bb8c-e02847dadd71 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:34.184445 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ade4e-8df5-40d2-9876-f4f560f90055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:34.185172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ade4e-8df5-40d2-9876-f4f560f90055 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:34.185460 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ade4e-8df5-40d2-9876-f4f560f90055 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:34.185732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ade4e-8df5-40d2-9876-f4f560f90055 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:34.186441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ade4e-8df5-40d2-9876-f4f560f90055 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:34.186746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ade4e-8df5-40d2-9876-f4f560f90055 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:34.188115 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ade4e-8df5-40d2-9876-f4f560f90055 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:34.197327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 392/778] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Jun 7 08:20:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:34.199661 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bc2d2-45b5-4c66-8572-39bc4c4fa07d None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:34.361982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ace14-00c7-4e59-9b4e-f76e151d7771 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:34.362481 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ace14-00c7-4e59-9b4e-f76e151d7771 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:34.362481 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ace14-00c7-4e59-9b4e-f76e151d7771 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:34.364041 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 393/779] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:20:34.365757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bc2d2-45b5-4c66-8572-39bc4c4fa07d None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:34.373043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddba5f-834d-4925-a56b-8ee8d89294ab None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:34.373339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddba5f-834d-4925-a56b-8ee8d89294ab None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:34.373618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddba5f-834d-4925-a56b-8ee8d89294ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:34.403917 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d0ddba5f-834d-4925-a56b-8ee8d89294ab None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:34.406267 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d0ddba5f-834d-4925-a56b-8ee8d89294ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:34.423462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64619b-0d49-4281-90c2-8c32e86574ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:34.424732 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64619b-0d49-4281-90c2-8c32e86574ae None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:34.425272 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64619b-0d49-4281-90c2-8c32e86574ae None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:34.425821 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64619b-0d49-4281-90c2-8c32e86574ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:34.426829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64619b-0d49-4281-90c2-8c32e86574ae None admin] RBAC: Adding query filter params (interface=public, service_id=2626e58d5d404d04a29763db90a38ff9, region_id=RegionOne) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:34.428459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64619b-0d49-4281-90c2-8c32e86574ae None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:34.430069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64619b-0d49-4281-90c2-8c32e86574ae None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:34.440535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 387/780] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Jun 7 08:20:34 2022] GET /identity/v3/endpoints?service_id=2626e58d5d404d04a29763db90a38ff9&interface=public®ion_id=RegionOne => generated 194 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:34.442282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e03592-e8b3-4476-950f-24a16c9d513a None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:34.442793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e03592-e8b3-4476-950f-24a16c9d513a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:34.443266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e03592-e8b3-4476-950f-24a16c9d513a None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:34.445203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 388/781] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:34.446178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e03592-e8b3-4476-950f-24a16c9d513a None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:34.460789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:34.501951 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 394/782] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:34.503625 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce648482-1788-47d6-9d8a-44d416ada271 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:34.504195 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce648482-1788-47d6-9d8a-44d416ada271 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:34.504616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce648482-1788-47d6-9d8a-44d416ada271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:34.527102 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ce648482-1788-47d6-9d8a-44d416ada271 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:34.528772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ce648482-1788-47d6-9d8a-44d416ada271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:34.588534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e03592-e8b3-4476-950f-24a16c9d513a None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:34.602253 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 395/783] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:34.609014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce648482-1788-47d6-9d8a-44d416ada271 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:34.632199 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e5559-e8b3-41a9-afb6-3a8db76acfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:34.633380 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e5559-e8b3-41a9-afb6-3a8db76acfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:34.633773 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e5559-e8b3-41a9-afb6-3a8db76acfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:34.634145 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e5559-e8b3-41a9-afb6-3a8db76acfb1 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:34.635484 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e5559-e8b3-41a9-afb6-3a8db76acfb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:34.636666 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e5559-e8b3-41a9-afb6-3a8db76acfb1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:34.651722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 389/784] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Jun 7 08:20:34 2022] GET /identity/v3/services => generated 772 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:34.651722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e5559-e8b3-41a9-afb6-3a8db76acfb1 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:34.656087 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:34.802357 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:34.810399 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce648482-1788-47d6-9d8a-44d416ada271 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:34.852595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e5559-e8b3-41a9-afb6-3a8db76acfb1 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:34.901192 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb920ed8-2e14-4ba0-8213-58db7a28eec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:34.902433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb920ed8-2e14-4ba0-8213-58db7a28eec8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:34.902782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded023e-5d38-4aa7-87eb-47e8ccb2cd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:34.903222 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb920ed8-2e14-4ba0-8213-58db7a28eec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:34.903640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded023e-5d38-4aa7-87eb-47e8ccb2cd7a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:34.903923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded023e-5d38-4aa7-87eb-47e8ccb2cd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:34.904198 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded023e-5d38-4aa7-87eb-47e8ccb2cd7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:34.904300 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb920ed8-2e14-4ba0-8213-58db7a28eec8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:34.917557 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb920ed8-2e14-4ba0-8213-58db7a28eec8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:34.929387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-bb920ed8-2e14-4ba0-8213-58db7a28eec8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:34.929387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:34.929387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:34.929387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:34.929387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:34.929387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:34.929387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:34.929387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:34.929387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:34.929387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:34.929387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:34.929387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:34.929387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:34.929387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:34.929387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:34.929387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:34.930343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:34.930343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:34.930343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:34.930343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:34.930343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:34.930343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:34.930343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:34.930343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:34.930343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:34.930343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:34.930343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:34.930343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:34.930343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:34.930343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:34.930343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:34.930343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:34.931179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:34.931179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:34.931179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:34.931179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:34.931179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:34.931179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:34.931179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:34.931179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:34.931179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:34.931179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:34.931179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:34.931179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:34.933270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded023e-5d38-4aa7-87eb-47e8ccb2cd7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:34.933988 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 390/785] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:20:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:34.942616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9ded023e-5d38-4aa7-87eb-47e8ccb2cd7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:34.942616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:34.942616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:34.942616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:34.942616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:34.942616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:34.942616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:34.942616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:34.942616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:34.942616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:34.942616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:34.942616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:34.942616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:34.942616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:34.942616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:34.942616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:34.943817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:34.943817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:34.943817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:34.943817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:34.943817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:34.943817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:34.943817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:34.943817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:34.943817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:34.943817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:34.943817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:34.943817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:34.943817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:34.943817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:34.943817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:34.943817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:34.944639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:34.944639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:34.944639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:34.944639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:34.944639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:34.944639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:34.944639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:34.944639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:34.944639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:34.944639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:34.944639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:34.944639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:34.944639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 396/786] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:34.946706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb920ed8-2e14-4ba0-8213-58db7a28eec8 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:34.951633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded023e-5d38-4aa7-87eb-47e8ccb2cd7a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:35.048152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:35.065345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded023e-5d38-4aa7-87eb-47e8ccb2cd7a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:35.084253 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb920ed8-2e14-4ba0-8213-58db7a28eec8 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:35.105951 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0fd66-9159-4962-8eb1-5e326c527d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:35.106673 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0fd66-9159-4962-8eb1-5e326c527d47 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:35.107455 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0fd66-9159-4962-8eb1-5e326c527d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:35.107730 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0fd66-9159-4962-8eb1-5e326c527d47 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:35.108472 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0fd66-9159-4962-8eb1-5e326c527d47 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:35.108776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0fd66-9159-4962-8eb1-5e326c527d47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:35.110225 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0fd66-9159-4962-8eb1-5e326c527d47 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:35.123274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 397/787] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:20:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:35.134020 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0fd66-9159-4962-8eb1-5e326c527d47 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:35.137340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9616f5-c059-4b94-a6b4-43ae98a80be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:35.138236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9616f5-c059-4b94-a6b4-43ae98a80be8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:35.138661 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9616f5-c059-4b94-a6b4-43ae98a80be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:35.139081 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9616f5-c059-4b94-a6b4-43ae98a80be8 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:35.139958 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9616f5-c059-4b94-a6b4-43ae98a80be8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:35.140393 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9616f5-c059-4b94-a6b4-43ae98a80be8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:35.142067 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9616f5-c059-4b94-a6b4-43ae98a80be8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:35.144570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:35.159718 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 391/788] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:35.161580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9616f5-c059-4b94-a6b4-43ae98a80be8 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:35.303338 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9616f5-c059-4b94-a6b4-43ae98a80be8 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:35.342473 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0fd66-9159-4962-8eb1-5e326c527d47 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:35.372570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bab39d-2638-4b6c-8ae1-ae886bb2148a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:35.373386 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bab39d-2638-4b6c-8ae1-ae886bb2148a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:35.373663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bab39d-2638-4b6c-8ae1-ae886bb2148a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:35.373933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bab39d-2638-4b6c-8ae1-ae886bb2148a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:35.392388 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043e4c4-ae44-4ff5-9524-76694e9ab740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:35.393488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043e4c4-ae44-4ff5-9524-76694e9ab740 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:35.393776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043e4c4-ae44-4ff5-9524-76694e9ab740 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:35.394055 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043e4c4-ae44-4ff5-9524-76694e9ab740 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:35.405535 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043e4c4-ae44-4ff5-9524-76694e9ab740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:35.406994 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043e4c4-ae44-4ff5-9524-76694e9ab740 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:35.407939 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bab39d-2638-4b6c-8ae1-ae886bb2148a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:35.409512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bab39d-2638-4b6c-8ae1-ae886bb2148a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:35.427097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:35.429390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-0043e4c4-ae44-4ff5-9524-76694e9ab740 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:35.429390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:35.429390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:35.429390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:35.429390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:35.429390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:35.429390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:35.429390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:35.429390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:35.429390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:35.429390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:35.429390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:35.429390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:35.429390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:35.429390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:35.430429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:35.430429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:35.430429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:35.430429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:35.430429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:35.430429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:35.430429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:35.430429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:35.430429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:35.430429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:35.430429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:35.430429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:35.430429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:35.430429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:35.430429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:35.430429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:35.430429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:35.431518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:35.431518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:35.431518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:35.431518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:35.432598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 398/789] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:35.437459 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043e4c4-ae44-4ff5-9524-76694e9ab740 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:35.448167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-f5bab39d-2638-4b6c-8ae1-ae886bb2148a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:35.448167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:35.448167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:35.448167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:35.448167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:35.448167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:35.448167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:35.448167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:35.448167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:35.448167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:35.448167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:35.448167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:35.448167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:35.448167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:35.448167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:35.449410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:35.449410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:35.449410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:35.449410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:35.449410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:35.449410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:35.449410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:35.449410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:35.449410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:35.449410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:35.449410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:35.449410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:35.449410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:35.449410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:35.449410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:35.449410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:35.449410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:35.451188 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:35.451188 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:35.451188 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:35.451188 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:35.453010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 392/790] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:35.457869 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bab39d-2638-4b6c-8ae1-ae886bb2148a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:35.581373 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:35.598103 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043e4c4-ae44-4ff5-9524-76694e9ab740 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:35.663381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fdda8-c736-4fc5-a673-0b852aa9d943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:35.664556 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fdda8-c736-4fc5-a673-0b852aa9d943 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:35.665766 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fdda8-c736-4fc5-a673-0b852aa9d943 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:35.666411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fdda8-c736-4fc5-a673-0b852aa9d943 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:35.667327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624fdda8-c736-4fc5-a673-0b852aa9d943 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:35.668606 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624fdda8-c736-4fc5-a673-0b852aa9d943 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:35.670458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624fdda8-c736-4fc5-a673-0b852aa9d943 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:35.685700 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bab39d-2638-4b6c-8ae1-ae886bb2148a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:35.694904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 399/791] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:35.696355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fdda8-c736-4fc5-a673-0b852aa9d943 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:35.728551 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71694699-e985-4c6f-bc05-1df50bc99b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:35.729430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71694699-e985-4c6f-bc05-1df50bc99b2f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:35.729715 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71694699-e985-4c6f-bc05-1df50bc99b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:35.729988 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71694699-e985-4c6f-bc05-1df50bc99b2f None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:35.733017 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71694699-e985-4c6f-bc05-1df50bc99b2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:35.733017 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71694699-e985-4c6f-bc05-1df50bc99b2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:35.733017 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71694699-e985-4c6f-bc05-1df50bc99b2f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:35.745896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 393/792] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:35.751627 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71694699-e985-4c6f-bc05-1df50bc99b2f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:35.811534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:35.889073 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fdda8-c736-4fc5-a673-0b852aa9d943 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:35.922772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd19b3-c487-4ba9-87e6-e49ba611685a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:35.923835 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd19b3-c487-4ba9-87e6-e49ba611685a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:35.924112 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd19b3-c487-4ba9-87e6-e49ba611685a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:35.924380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd19b3-c487-4ba9-87e6-e49ba611685a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:35.932089 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd19b3-c487-4ba9-87e6-e49ba611685a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:35.933598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd19b3-c487-4ba9-87e6-e49ba611685a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:35.943232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71694699-e985-4c6f-bc05-1df50bc99b2f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:35.948989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-87fd19b3-c487-4ba9-87e6-e49ba611685a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:20:35.948989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:35.948989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:35.948989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:35.948989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:35.948989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:35.948989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:35.948989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:35.948989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:35.948989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:35.948989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:35.948989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:35.948989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:35.948989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:35.948989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:35.948989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:35.949945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:35.949945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:35.949945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:35.949945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:35.949945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:35.949945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:35.949945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:35.949945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:35.949945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:35.949945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:35.949945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:35.949945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:35.949945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:35.949945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:35.949945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:35.949945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:35.949945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:35.949945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:35.951613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:35.951613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:35.951613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:35.951613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:35.951613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:35.951613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:35.951613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:35.951613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:35.951613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:20:35.951613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:35.951613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 400/793] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jun 7 08:20:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:35.952197 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853ca67-691a-40cd-9003-7368e33b1a2d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:35.952354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853ca67-691a-40cd-9003-7368e33b1a2d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:35.952645 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853ca67-691a-40cd-9003-7368e33b1a2d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:35.953925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:35.958644 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 401/794] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:35.959413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853ca67-691a-40cd-9003-7368e33b1a2d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:35.980676 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ca31a-0c1e-4caa-882e-e4c6f8727546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:35.982445 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ca31a-0c1e-4caa-882e-e4c6f8727546 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:35.986203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ca31a-0c1e-4caa-882e-e4c6f8727546 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:35.986203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ca31a-0c1e-4caa-882e-e4c6f8727546 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:35.994025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ca31a-0c1e-4caa-882e-e4c6f8727546 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:35.995716 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ca31a-0c1e-4caa-882e-e4c6f8727546 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:36.021075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-6f5ca31a-0c1e-4caa-882e-e4c6f8727546 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:20:36.021075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:36.021075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:36.021075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:36.021075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:36.021075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:36.021075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:36.021075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:36.021075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:36.021075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:36.021075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:36.021075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:36.021075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:36.021075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:36.021075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:36.021075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:36.022087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:36.022087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:36.022087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:36.022087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:36.022087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:36.022087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:36.022087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:36.022087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:36.022087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:36.022087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:36.022087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:36.022087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:36.022087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:36.022087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:36.022087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:36.022087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:36.022087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:36.022087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:36.023041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:36.023041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:36.023041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:36.023041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:36.023041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:36.023041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:36.023041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:36.023041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:36.023041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:20:36.023041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:36.026039 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 394/795] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Jun 7 08:20:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:36.027661 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271700c8-11ad-4c14-a12a-c29359cccaab None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:36.028083 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271700c8-11ad-4c14-a12a-c29359cccaab None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:36.028525 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271700c8-11ad-4c14-a12a-c29359cccaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:36.047029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-271700c8-11ad-4c14-a12a-c29359cccaab None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:36.048233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-271700c8-11ad-4c14-a12a-c29359cccaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:36.074404 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853ca67-691a-40cd-9003-7368e33b1a2d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:36.092895 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:36.110101 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b7baa-cce7-4875-a60a-a0e0bc94dbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:36.111906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b7baa-cce7-4875-a60a-a0e0bc94dbc7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:36.111906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b7baa-cce7-4875-a60a-a0e0bc94dbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:36.111906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b7baa-cce7-4875-a60a-a0e0bc94dbc7 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:36.112310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b7baa-cce7-4875-a60a-a0e0bc94dbc7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:36.112616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b7baa-cce7-4875-a60a-a0e0bc94dbc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:36.114091 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b7baa-cce7-4875-a60a-a0e0bc94dbc7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:36.116361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 395/796] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:36.117226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271700c8-11ad-4c14-a12a-c29359cccaab None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:36.141347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 402/797] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Jun 7 08:20:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:36.142717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a7875-ca62-4cc3-8988-b7319b8da3f5 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:36.143003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a7875-ca62-4cc3-8988-b7319b8da3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:36.143282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a7875-ca62-4cc3-8988-b7319b8da3f5 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:36.144553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 403/798] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:36.145680 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a7875-ca62-4cc3-8988-b7319b8da3f5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:36.233260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:36.236963 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271700c8-11ad-4c14-a12a-c29359cccaab None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:36.265410 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a7875-ca62-4cc3-8988-b7319b8da3f5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:36.271649 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33f599-b0ad-43a2-a14b-4a89657e7545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:36.272347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33f599-b0ad-43a2-a14b-4a89657e7545 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:36.272629 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33f599-b0ad-43a2-a14b-4a89657e7545 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:36.272904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33f599-b0ad-43a2-a14b-4a89657e7545 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:36.274262 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33f599-b0ad-43a2-a14b-4a89657e7545 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:36.274262 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33f599-b0ad-43a2-a14b-4a89657e7545 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:36.275566 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33f599-b0ad-43a2-a14b-4a89657e7545 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:36.295560 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 396/799] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Jun 7 08:20:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:36.296258 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33f599-b0ad-43a2-a14b-4a89657e7545 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:36.307163 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e37883-19a1-4667-ab9f-533ef14883b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:36.308003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e37883-19a1-4667-ab9f-533ef14883b9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:36.308287 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e37883-19a1-4667-ab9f-533ef14883b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:36.308595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e37883-19a1-4667-ab9f-533ef14883b9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:36.309354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e37883-19a1-4667-ab9f-533ef14883b9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:36.310707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e37883-19a1-4667-ab9f-533ef14883b9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:36.323746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-36e37883-19a1-4667-ab9f-533ef14883b9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 08:20:36.323746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:36.323746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:20:36.323746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:20:36.323746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:20:36.323746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:20:36.323746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:20:36.323746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:36.323746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 08:20:36.323746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:36.323746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:36.323746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:36.323746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:36.323746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:36.323746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:36.323746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:36.323746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:36.324767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:36.324767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:36.324767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:36.324767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:36.324767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:36.324767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:36.324767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:20:36.324767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:20:36.324767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:20:36.324767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:20:36.324767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:36.324767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:36.324767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:36.324767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:36.324767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:36.324767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:36.326595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:36.326595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:36.326595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:36.326595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:36.326595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:36.326595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:36.326595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:36.326595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:36.326595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:36.326595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:36.326595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:20:36.326595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:36.326595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 08:20:36.326595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:36.326595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 404/800] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:36 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:36.327454 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894f692-642d-4a78-9323-4a5f7d7a7c8f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:36.327454 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894f692-642d-4a78-9323-4a5f7d7a7c8f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:36.327454 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894f692-642d-4a78-9323-4a5f7d7a7c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:36.344268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3894f692-642d-4a78-9323-4a5f7d7a7c8f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:36.345502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3894f692-642d-4a78-9323-4a5f7d7a7c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:36.381033 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:36.421033 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 405/801] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:36.421033 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894f692-642d-4a78-9323-4a5f7d7a7c8f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:36.425462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33f599-b0ad-43a2-a14b-4a89657e7545 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:36.462839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d0124-e66b-4c11-a50b-864e28afd25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:36.463719 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d0124-e66b-4c11-a50b-864e28afd25c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:36.464001 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d0124-e66b-4c11-a50b-864e28afd25c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:36.464277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d0124-e66b-4c11-a50b-864e28afd25c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:36.475461 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d0124-e66b-4c11-a50b-864e28afd25c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:36.477065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d0124-e66b-4c11-a50b-864e28afd25c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:36.496227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-034d0124-e66b-4c11-a50b-864e28afd25c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:36.496227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:36.496227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:36.496227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:36.496227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:36.496227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:36.496227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:36.496227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:36.496227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:36.496227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:36.496227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:36.496227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:36.496227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:36.496227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:36.496227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:36.497153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:36.497153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:36.497153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:36.497153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:36.497153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:36.497153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:36.497153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:36.497153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:36.497153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:36.497153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:36.497153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:36.497153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:36.497153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:36.497153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:36.497153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:36.497153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:36.497153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:36.498586 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:36.498586 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:36.498586 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:36.498586 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:36.498586 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 397/802] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:36.498869 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d0124-e66b-4c11-a50b-864e28afd25c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:36.511932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:36.522532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894f692-642d-4a78-9323-4a5f7d7a7c8f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:36.559123 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a534c63-b41f-4430-874e-4985f3831135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:36.560310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a534c63-b41f-4430-874e-4985f3831135 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:36.560589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a534c63-b41f-4430-874e-4985f3831135 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:36.560866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a534c63-b41f-4430-874e-4985f3831135 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:36.604242 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d0124-e66b-4c11-a50b-864e28afd25c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:36.605859 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:36.637698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a534c63-b41f-4430-874e-4985f3831135 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:36.639656 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa4d78-d43f-4007-b22c-d533551d2bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:36.639807 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a534c63-b41f-4430-874e-4985f3831135 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:36.640356 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa4d78-d43f-4007-b22c-d533551d2bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:36.640640 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa4d78-d43f-4007-b22c-d533551d2bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:36.640913 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa4d78-d43f-4007-b22c-d533551d2bf7 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:36.641660 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa4d78-d43f-4007-b22c-d533551d2bf7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:36.641970 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa4d78-d43f-4007-b22c-d533551d2bf7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:36.643321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa4d78-d43f-4007-b22c-d533551d2bf7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:36.647821 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 398/803] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jun 7 08:20:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:36.648606 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa4d78-d43f-4007-b22c-d533551d2bf7 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:36.717982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:36.748526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa4d78-d43f-4007-b22c-d533551d2bf7 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:36.753309 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 406/804] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:20:36.754420 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a534c63-b41f-4430-874e-4985f3831135 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:36.784838 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138fc79-2163-4d94-b227-ee8f18ce5d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:36.785787 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138fc79-2163-4d94-b227-ee8f18ce5d80 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:36.786070 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138fc79-2163-4d94-b227-ee8f18ce5d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:36.786372 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138fc79-2163-4d94-b227-ee8f18ce5d80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:36.797109 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5138fc79-2163-4d94-b227-ee8f18ce5d80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:36.805737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-5138fc79-2163-4d94-b227-ee8f18ce5d80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:36.805737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:36.805737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:36.805737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:36.805737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:36.805737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:36.805737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:36.805737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:36.805737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:36.805737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:36.805737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:36.805737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:36.805737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:36.805737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:36.805737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:36.805737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:36.806679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:36.806679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:36.806679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:36.806679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:36.806679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:36.806679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:36.806679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:36.806679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:36.806679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:36.806679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:36.806679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:36.806679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:36.806679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:36.806679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:36.806679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:36.806679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:36.807526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:36.807526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:36.807526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:36.807526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:36.807526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:36.807526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:36.807526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:36.807526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:36.807526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:36.807526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:36.807526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:36.807526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:36.807526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 399/805] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:20:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:36.809104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138fc79-2163-4d94-b227-ee8f18ce5d80 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:36.833367 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:36.863908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a534c63-b41f-4430-874e-4985f3831135 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:36.899804 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fd2fe-7c3e-4819-87d3-b358b047992a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:36.900466 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fd2fe-7c3e-4819-87d3-b358b047992a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:36.900746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fd2fe-7c3e-4819-87d3-b358b047992a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:36.901039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fd2fe-7c3e-4819-87d3-b358b047992a None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:36.901761 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40fd2fe-7c3e-4819-87d3-b358b047992a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:36.902064 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40fd2fe-7c3e-4819-87d3-b358b047992a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:36.903461 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40fd2fe-7c3e-4819-87d3-b358b047992a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:36.915357 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 407/806] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:36.916153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fd2fe-7c3e-4819-87d3-b358b047992a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:36.916760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138fc79-2163-4d94-b227-ee8f18ce5d80 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:36.949863 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:36.954946 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ef254-c364-4b36-9374-7fab5fc4a7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:36.955603 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ef254-c364-4b36-9374-7fab5fc4a7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:36.955884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ef254-c364-4b36-9374-7fab5fc4a7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:36.956153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ef254-c364-4b36-9374-7fab5fc4a7c3 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:36.956865 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ef254-c364-4b36-9374-7fab5fc4a7c3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:36.957199 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ef254-c364-4b36-9374-7fab5fc4a7c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:36.958581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ef254-c364-4b36-9374-7fab5fc4a7c3 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:36.963338 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 400/807] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jun 7 08:20:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:36.964116 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ef254-c364-4b36-9374-7fab5fc4a7c3 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:37.031091 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fd2fe-7c3e-4819-87d3-b358b047992a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:37.062122 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:37.067499 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169c2b0-005f-4bb8-9c99-4d1e225757e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:37.068387 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169c2b0-005f-4bb8-9c99-4d1e225757e9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:37.068674 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169c2b0-005f-4bb8-9c99-4d1e225757e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:37.068936 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169c2b0-005f-4bb8-9c99-4d1e225757e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:37.071016 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ef254-c364-4b36-9374-7fab5fc4a7c3 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:37.078948 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169c2b0-005f-4bb8-9c99-4d1e225757e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:37.080404 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169c2b0-005f-4bb8-9c99-4d1e225757e9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:37.098477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-4169c2b0-005f-4bb8-9c99-4d1e225757e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:37.098477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:37.098477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:37.098477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:37.098477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:37.098477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:37.098477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:37.098477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:37.098477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:37.098477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:37.098477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:37.098477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:37.098477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:37.098477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:37.098477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:37.099438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:37.099438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:37.099438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:37.099438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:37.099438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:37.099438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:37.099438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:37.099438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:37.099438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:37.099438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:37.099438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:37.099438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:37.099438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:37.099438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:37.099438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:37.099438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:37.099438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:37.100329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:37.100329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:37.100329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:37.100329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:37.104198 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 408/808] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:37.105057 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169c2b0-005f-4bb8-9c99-4d1e225757e9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:37.108755 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fd715-360d-4756-a2d4-a74d1c1563f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:37.109738 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fd715-360d-4756-a2d4-a74d1c1563f2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:37.109738 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fd715-360d-4756-a2d4-a74d1c1563f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:37.109988 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fd715-360d-4756-a2d4-a74d1c1563f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:37.110735 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fd715-360d-4756-a2d4-a74d1c1563f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:37.111042 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fd715-360d-4756-a2d4-a74d1c1563f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:37.112414 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fd715-360d-4756-a2d4-a74d1c1563f2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:37.121493 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 401/809] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:37.122055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fd715-360d-4756-a2d4-a74d1c1563f2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:37.185438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:37.225858 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169c2b0-005f-4bb8-9c99-4d1e225757e9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:37.228719 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fd715-360d-4756-a2d4-a74d1c1563f2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:37.263429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6e4f4-d565-4ded-a9e6-1f8626660ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:37.264220 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6e4f4-d565-4ded-a9e6-1f8626660ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:37.264449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046c98e-8e43-4403-9606-44ab9384fcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:37.264567 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6e4f4-d565-4ded-a9e6-1f8626660ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:37.264765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6e4f4-d565-4ded-a9e6-1f8626660ea2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:37.265131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046c98e-8e43-4403-9606-44ab9384fcef None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:37.265415 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046c98e-8e43-4403-9606-44ab9384fcef None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:37.265696 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046c98e-8e43-4403-9606-44ab9384fcef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:37.266425 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046c98e-8e43-4403-9606-44ab9384fcef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:37.268330 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046c98e-8e43-4403-9606-44ab9384fcef None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:37.271718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6e4f4-d565-4ded-a9e6-1f8626660ea2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:37.273165 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6e4f4-d565-4ded-a9e6-1f8626660ea2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:37.291576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-85f6e4f4-d565-4ded-a9e6-1f8626660ea2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:37.291576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:37.291576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:37.291576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:37.291576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:37.291576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:37.291576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:37.291576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:37.291576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:37.291576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:37.291576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:37.291576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:37.291576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:37.291576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:37.291576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:37.292537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:37.292537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:37.292537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:37.292537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:37.292537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:37.292537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:37.292537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:37.292537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:37.292537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:37.292537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:37.292537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:37.292537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:37.292537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:37.292537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:37.292537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:37.292537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:37.292537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:37.293462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:37.293462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:37.293462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:37.293462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:37.293462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:37.293462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:37.293462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:37.293462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:37.293462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 409/810] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:37.293462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6e4f4-d565-4ded-a9e6-1f8626660ea2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:37.335507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:37.367223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 402/811] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jun 7 08:20:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:37.368101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046c98e-8e43-4403-9606-44ab9384fcef None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:37.394896 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6e4f4-d565-4ded-a9e6-1f8626660ea2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:37.454339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f2e36-6d6c-4a89-9ddb-583394b1cd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:37.455783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f2e36-6d6c-4a89-9ddb-583394b1cd77 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:37.456116 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f2e36-6d6c-4a89-9ddb-583394b1cd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:37.457038 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f2e36-6d6c-4a89-9ddb-583394b1cd77 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:37.457265 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f2e36-6d6c-4a89-9ddb-583394b1cd77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:37.457574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f2e36-6d6c-4a89-9ddb-583394b1cd77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:37.459090 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f2e36-6d6c-4a89-9ddb-583394b1cd77 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:37.478905 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 410/812] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:37.478905 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f2e36-6d6c-4a89-9ddb-583394b1cd77 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:37.498302 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046c98e-8e43-4403-9606-44ab9384fcef None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:37.513816 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:37.540961 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55361aef-55bd-47ae-abb9-4ed077790e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:37.541979 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55361aef-55bd-47ae-abb9-4ed077790e77 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:37.542189 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55361aef-55bd-47ae-abb9-4ed077790e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:37.542573 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55361aef-55bd-47ae-abb9-4ed077790e77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:37.557102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55361aef-55bd-47ae-abb9-4ed077790e77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:37.558687 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55361aef-55bd-47ae-abb9-4ed077790e77 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:37.576746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-55361aef-55bd-47ae-abb9-4ed077790e77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:37.576746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:37.576746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:37.576746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:37.576746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:37.576746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:37.576746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:37.576746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:37.576746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:37.576746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:37.576746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:37.576746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:37.576746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:37.576746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:37.576746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:37.577743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:37.577743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:37.577743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:37.577743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:37.577743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:37.577743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:37.577743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:37.577743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:37.577743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:37.577743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:37.577743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:37.577743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:37.577743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:37.577743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:37.577743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:37.577743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:37.577743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:37.579250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:37.579250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:37.579250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:37.579250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:37.579250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 403/813] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:37.579647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab33c3-dbd3-43e9-9de5-a60f9e4c955d None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:37.579932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab33c3-dbd3-43e9-9de5-a60f9e4c955d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:37.580213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab33c3-dbd3-43e9-9de5-a60f9e4c955d None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:37.581499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 404/814] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:37.582182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab33c3-dbd3-43e9-9de5-a60f9e4c955d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:37.596270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:37.602851 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f2e36-6d6c-4a89-9ddb-583394b1cd77 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:37.649899 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd4ddc-64f7-42d3-be01-068a2b77d684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:37.650559 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd4ddc-64f7-42d3-be01-068a2b77d684 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:37.650837 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd4ddc-64f7-42d3-be01-068a2b77d684 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:37.651113 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd4ddc-64f7-42d3-be01-068a2b77d684 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:37.651917 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd4ddc-64f7-42d3-be01-068a2b77d684 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:37.652223 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd4ddc-64f7-42d3-be01-068a2b77d684 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:37.654345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd4ddc-64f7-42d3-be01-068a2b77d684 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:37.665554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 411/815] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:37.666827 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdbdba-bfd7-4f51-b129-6c8dc1642f1f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:37.667112 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdbdba-bfd7-4f51-b129-6c8dc1642f1f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:37.667387 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdbdba-bfd7-4f51-b129-6c8dc1642f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:37.683214 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-80cdbdba-bfd7-4f51-b129-6c8dc1642f1f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:37.684399 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-80cdbdba-bfd7-4f51-b129-6c8dc1642f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:37.691871 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab33c3-dbd3-43e9-9de5-a60f9e4c955d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:37.724750 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641f3ab-6069-4a44-a5f0-8ae2b90192db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:37.725572 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641f3ab-6069-4a44-a5f0-8ae2b90192db None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:37.725859 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641f3ab-6069-4a44-a5f0-8ae2b90192db None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:37.726141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641f3ab-6069-4a44-a5f0-8ae2b90192db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:37.732604 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641f3ab-6069-4a44-a5f0-8ae2b90192db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:37.734087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641f3ab-6069-4a44-a5f0-8ae2b90192db None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:37.747541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 412/816] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:37.747896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-1641f3ab-6069-4a44-a5f0-8ae2b90192db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:37.747896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:37.747896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:37.747896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:37.747896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:37.747896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:37.747896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:37.747896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:37.747896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:37.747896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:37.747896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:37.747896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:37.747896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:37.747896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:37.747896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:37.748756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:37.748756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:37.748756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:37.748756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:37.748756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:37.748756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:37.748756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:37.748756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:37.748756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:37.748756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:37.748756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:37.748756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:37.748756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:37.748756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:37.748756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:37.748756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:37.748756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:37.749702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:37.749702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:37.749702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:37.749702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:37.749702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:37.749702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:37.749702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:37.749702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:37.750142 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdbdba-bfd7-4f51-b129-6c8dc1642f1f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:37.750213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 405/817] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:37.750288 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641f3ab-6069-4a44-a5f0-8ae2b90192db None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:37.782442 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:37.837415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:37.849119 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdbdba-bfd7-4f51-b129-6c8dc1642f1f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:37.853995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641f3ab-6069-4a44-a5f0-8ae2b90192db None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:37.884263 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a70cd-4041-4981-bc1f-1a3976db717d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:37.884925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a70cd-4041-4981-bc1f-1a3976db717d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:37.885232 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a70cd-4041-4981-bc1f-1a3976db717d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:37.885510 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a70cd-4041-4981-bc1f-1a3976db717d None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:37.886287 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a70cd-4041-4981-bc1f-1a3976db717d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:37.886533 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a70cd-4041-4981-bc1f-1a3976db717d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:37.887707 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4db87-4cf0-49f3-be17-7948a85fcdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:37.888487 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a70cd-4041-4981-bc1f-1a3976db717d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:37.889119 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4db87-4cf0-49f3-be17-7948a85fcdcb None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:37.889401 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4db87-4cf0-49f3-be17-7948a85fcdcb None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:37.889677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4db87-4cf0-49f3-be17-7948a85fcdcb None admin] PATH_INFO: `/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/bbf0ce396885472bb5d225aa009043f2/roles/a04f2ab5f1234bd6b1dc4c18b4210a89` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:37.899472 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 413/818] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:37.900255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a70cd-4041-4981-bc1f-1a3976db717d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:37.908242 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba4db87-4cf0-49f3-be17-7948a85fcdcb None admin] RBAC: Authorizing `identity:create_grant(project_id=afc8781e5f864d4a9374300ee9b5193b, user_id=bbf0ce396885472bb5d225aa009043f2, role_id=a04f2ab5f1234bd6b1dc4c18b4210a89)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:37.909738 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba4db87-4cf0-49f3-be17-7948a85fcdcb None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:37.972329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 406/819] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:20:37 2022] PUT /identity/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/bbf0ce396885472bb5d225aa009043f2/roles/a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:20:37.973143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4db87-4cf0-49f3-be17-7948a85fcdcb None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:37.998059 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a70cd-4041-4981-bc1f-1a3976db717d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:38.035934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9047f-e7c4-45c7-aa11-e79ed53aff52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:38.036620 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9047f-e7c4-45c7-aa11-e79ed53aff52 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:38.036900 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9047f-e7c4-45c7-aa11-e79ed53aff52 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:38.037213 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9047f-e7c4-45c7-aa11-e79ed53aff52 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:38.038022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a9047f-e7c4-45c7-aa11-e79ed53aff52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:38.038337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a9047f-e7c4-45c7-aa11-e79ed53aff52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:38.039812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a9047f-e7c4-45c7-aa11-e79ed53aff52 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:38.058452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 414/820] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:38.058452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9047f-e7c4-45c7-aa11-e79ed53aff52 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:38.070510 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:38.084940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:38.109747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4db87-4cf0-49f3-be17-7948a85fcdcb None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:38.153219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0f4de-f90d-4b50-bf61-d8d0dcdd2c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:38.155306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0f4de-f90d-4b50-bf61-d8d0dcdd2c0e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:38.155306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0f4de-f90d-4b50-bf61-d8d0dcdd2c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:38.155306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0f4de-f90d-4b50-bf61-d8d0dcdd2c0e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:38.155538 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0f4de-f90d-4b50-bf61-d8d0dcdd2c0e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:38.156842 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0f4de-f90d-4b50-bf61-d8d0dcdd2c0e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:38.170271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-10f0f4de-f90d-4b50-bf61-d8d0dcdd2c0e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 08:20:38.170271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:38.170271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:20:38.170271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:20:38.170271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:20:38.170271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:20:38.170271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:20:38.170271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:38.170271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 08:20:38.170271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:38.170271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:38.170271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:38.170271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:38.170271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:38.170271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:38.170271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:38.171321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:38.171321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:38.171321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:38.171321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:38.171321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:38.171321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:38.171321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:38.171321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:20:38.171321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:20:38.171321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:20:38.171321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:20:38.171321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:38.171321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:38.171321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:38.171321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:38.171321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:38.172622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:38.172622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:38.172622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:38.172622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:38.172622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:38.172622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:38.172622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:38.172622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:38.172622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:38.172622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:38.172622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:38.172622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:20:38.172622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:38.172622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 08:20:38.172622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:38.172622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 407/821] 10.222.0.17 () {56 vars in 1129 bytes} [Tue Jun 7 08:20:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:38.174017 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9047f-e7c4-45c7-aa11-e79ed53aff52 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:38.177989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0f4de-f90d-4b50-bf61-d8d0dcdd2c0e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:38.218577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0c8e2-8fca-46c7-acae-02ea29b95080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:38.220059 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0c8e2-8fca-46c7-acae-02ea29b95080 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:38.220352 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0c8e2-8fca-46c7-acae-02ea29b95080 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:38.220632 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0c8e2-8fca-46c7-acae-02ea29b95080 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:38.230406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0c8e2-8fca-46c7-acae-02ea29b95080 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:38.232621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0c8e2-8fca-46c7-acae-02ea29b95080 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:38.248594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1db0c8e2-8fca-46c7-acae-02ea29b95080 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:20:38.248594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:38.248594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:38.248594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:38.248594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:38.248594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:38.248594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:38.248594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:38.248594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:38.248594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:38.248594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:38.248594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:38.248594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:38.248594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:38.248594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:38.248594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:38.249803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:38.249803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:38.249803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:38.249803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:38.249803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:38.249803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:38.249803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:38.249803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:38.249803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:38.249803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:38.249803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:38.249803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:38.249803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:38.249803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:38.249803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:38.249803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:38.249803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:38.249803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:38.250992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:38.250992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:38.250992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:38.250992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:38.250992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:38.250992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:38.250992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:38.250992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:38.250992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:20:38.250992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:38.253186 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 415/822] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:20:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:38.255432 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0c8e2-8fca-46c7-acae-02ea29b95080 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:38.295528 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0f4de-f90d-4b50-bf61-d8d0dcdd2c0e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:38.325087 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:38.332141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9e3d4-7ec2-4d91-83bc-3bb59b4c5389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:38.332799 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9e3d4-7ec2-4d91-83bc-3bb59b4c5389 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:38.333118 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9e3d4-7ec2-4d91-83bc-3bb59b4c5389 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:38.333397 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9e3d4-7ec2-4d91-83bc-3bb59b4c5389 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:38.334226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9e3d4-7ec2-4d91-83bc-3bb59b4c5389 None admin] RBAC: Adding query filter params (role.id=8c63c00418a346b29f7db3f87a404e70, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=ee42568b059b49a198ed2c01d5efdafd) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:38.334544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9e3d4-7ec2-4d91-83bc-3bb59b4c5389 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:38.335980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9e3d4-7ec2-4d91-83bc-3bb59b4c5389 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:38.353305 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 408/823] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:38 2022] GET /identity/v3/role_assignments?user.id=ee42568b059b49a198ed2c01d5efdafd&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:38.353305 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9e3d4-7ec2-4d91-83bc-3bb59b4c5389 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:38.355375 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:38.374278 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0c8e2-8fca-46c7-acae-02ea29b95080 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:38.421370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e12f36-e04e-4171-924e-5c66aa2970ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:38.422072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e12f36-e04e-4171-924e-5c66aa2970ff None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:38.422863 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e12f36-e04e-4171-924e-5c66aa2970ff None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:38.423151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e12f36-e04e-4171-924e-5c66aa2970ff None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:38.424436 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e12f36-e04e-4171-924e-5c66aa2970ff None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:38.424747 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e12f36-e04e-4171-924e-5c66aa2970ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:38.426972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e12f36-e04e-4171-924e-5c66aa2970ff None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:38.434067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 416/824] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Jun 7 08:20:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:38.440194 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e12f36-e04e-4171-924e-5c66aa2970ff None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:38.475854 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9e3d4-7ec2-4d91-83bc-3bb59b4c5389 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:38.515651 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ffaa2-f3ee-4de8-b4c1-7f7a179d48c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:38.516341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ffaa2-f3ee-4de8-b4c1-7f7a179d48c0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:38.516623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ffaa2-f3ee-4de8-b4c1-7f7a179d48c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:38.516899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ffaa2-f3ee-4de8-b4c1-7f7a179d48c0 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:38.517807 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ffaa2-f3ee-4de8-b4c1-7f7a179d48c0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:38.518225 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ffaa2-f3ee-4de8-b4c1-7f7a179d48c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:38.519875 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ffaa2-f3ee-4de8-b4c1-7f7a179d48c0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:38.545536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 409/825] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Jun 7 08:20:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:38.547064 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ffaa2-f3ee-4de8-b4c1-7f7a179d48c0 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:38.576381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e12f36-e04e-4171-924e-5c66aa2970ff None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:38.618950 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:38.630254 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:38.630408 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097c2d9-b4c6-43a4-b4c1-48ecea1bd7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:38.630408 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097c2d9-b4c6-43a4-b4c1-48ecea1bd7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:38.630408 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097c2d9-b4c6-43a4-b4c1-48ecea1bd7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:38.630408 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097c2d9-b4c6-43a4-b4c1-48ecea1bd7a1 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:38.631258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097c2d9-b4c6-43a4-b4c1-48ecea1bd7a1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:38.631258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097c2d9-b4c6-43a4-b4c1-48ecea1bd7a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:38.632797 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097c2d9-b4c6-43a4-b4c1-48ecea1bd7a1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:38.644280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 417/826] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Jun 7 08:20:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:38.645451 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097c2d9-b4c6-43a4-b4c1-48ecea1bd7a1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:38.677843 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ffaa2-f3ee-4de8-b4c1-7f7a179d48c0 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:38.718228 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0a26e-529c-40ec-b369-e163f05a3d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:38.719025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0a26e-529c-40ec-b369-e163f05a3d5c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:38.719303 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0a26e-529c-40ec-b369-e163f05a3d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:38.719575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0a26e-529c-40ec-b369-e163f05a3d5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:38.730940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f0a26e-529c-40ec-b369-e163f05a3d5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:38.732430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f0a26e-529c-40ec-b369-e163f05a3d5c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:38.752257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-e8f0a26e-529c-40ec-b369-e163f05a3d5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:38.752257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:38.752257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:38.752257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:38.752257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:38.752257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:38.752257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:38.752257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:38.752257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:38.752257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:38.752257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:38.752257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:38.752257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:38.752257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:38.752257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:38.753218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:38.753218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:38.753218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:38.753218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:38.753218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:38.753218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:38.753218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:38.753218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:38.753218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:38.753218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:38.753218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:38.753218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:38.753218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:38.753218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:38.753218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:38.753218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:38.753218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:38.754087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:38.754087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:38.754087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:38.754087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:38.754087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 410/827] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:38.756690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097c2d9-b4c6-43a4-b4c1-48ecea1bd7a1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:38.759753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0a26e-529c-40ec-b369-e163f05a3d5c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:38.791969 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7f32e-d2d0-477d-b77d-e7e71ce25616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:38.792641 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7f32e-d2d0-477d-b77d-e7e71ce25616 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:38.792926 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7f32e-d2d0-477d-b77d-e7e71ce25616 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:38.793248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7f32e-d2d0-477d-b77d-e7e71ce25616 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:38.794004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7f32e-d2d0-477d-b77d-e7e71ce25616 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:38.795400 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7f32e-d2d0-477d-b77d-e7e71ce25616 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:38.866681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0a26e-529c-40ec-b369-e163f05a3d5c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:38.877349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 418/828] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jun 7 08:20:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:38.884449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:38.912644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8dd241-7c4d-42dd-bfad-9367f1b1e23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:38.913415 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8dd241-7c4d-42dd-bfad-9367f1b1e23c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:38.913706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8dd241-7c4d-42dd-bfad-9367f1b1e23c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:38.913984 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8dd241-7c4d-42dd-bfad-9367f1b1e23c None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:38.914717 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8dd241-7c4d-42dd-bfad-9367f1b1e23c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:38.915024 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8dd241-7c4d-42dd-bfad-9367f1b1e23c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:38.916430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8dd241-7c4d-42dd-bfad-9367f1b1e23c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:38.930263 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 411/829] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:38.936527 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7f32e-d2d0-477d-b77d-e7e71ce25616 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:38.996431 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:39.050919 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7f32e-d2d0-477d-b77d-e7e71ce25616 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:39.088716 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab706f3a-d126-4ecf-b557-438bac87899b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:39.089579 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab706f3a-d126-4ecf-b557-438bac87899b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:39.090631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab706f3a-d126-4ecf-b557-438bac87899b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:39.090939 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab706f3a-d126-4ecf-b557-438bac87899b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:39.099871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab706f3a-d126-4ecf-b557-438bac87899b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:39.101401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab706f3a-d126-4ecf-b557-438bac87899b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:39.116912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-ab706f3a-d126-4ecf-b557-438bac87899b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:39.116912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:39.116912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:39.116912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:39.116912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:39.116912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:39.116912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:39.116912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:39.116912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:39.116912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:39.116912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:39.116912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:39.116912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:39.116912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:39.116912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:39.117852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:39.117852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:39.117852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:39.117852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:39.117852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:39.117852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:39.117852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:39.117852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:39.117852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:39.117852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:39.117852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:39.117852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:39.117852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:39.117852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:39.117852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:39.117852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:39.117852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:39.118819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:39.118819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:39.118819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:39.118819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:39.118819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:39.118819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:39.118819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:39.118819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:39.118819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 419/830] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:39.124354 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8dd241-7c4d-42dd-bfad-9367f1b1e23c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:39.236621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8dd241-7c4d-42dd-bfad-9367f1b1e23c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:39.273706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30c94a-719c-40ef-8d30-cd34f4146c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:39.274426 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30c94a-719c-40ef-8d30-cd34f4146c81 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:39.274707 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30c94a-719c-40ef-8d30-cd34f4146c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:39.274977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30c94a-719c-40ef-8d30-cd34f4146c81 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:39.275762 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30c94a-719c-40ef-8d30-cd34f4146c81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:39.276067 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30c94a-719c-40ef-8d30-cd34f4146c81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:39.277533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30c94a-719c-40ef-8d30-cd34f4146c81 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:39.291161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 412/831] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:39.297568 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab706f3a-d126-4ecf-b557-438bac87899b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:39.401948 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab706f3a-d126-4ecf-b557-438bac87899b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:39.414770 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:39.436803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408983bc-1747-4894-b6ac-065a69e1d459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:39.437716 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408983bc-1747-4894-b6ac-065a69e1d459 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:39.437995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408983bc-1747-4894-b6ac-065a69e1d459 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:39.438268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408983bc-1747-4894-b6ac-065a69e1d459 None admin] PATH_INFO: `/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/334dc47ae2e643ccb44d25c6b5906f14/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:39.440345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408983bc-1747-4894-b6ac-065a69e1d459 None admin] RBAC: Authorizing `identity:create_grant(project_id=afc8781e5f864d4a9374300ee9b5193b, user_id=334dc47ae2e643ccb44d25c6b5906f14, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:39.441878 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408983bc-1747-4894-b6ac-065a69e1d459 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:39.522843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 420/832] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:20:39 2022] PUT /identity/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/334dc47ae2e643ccb44d25c6b5906f14/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:20:40.268649 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918527c-822a-4132-871a-98451edff0ec None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:40.269205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918527c-822a-4132-871a-98451edff0ec None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:40.269488 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918527c-822a-4132-871a-98451edff0ec None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:40.271207 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 413/833] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:40.276856 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95985f7d-acad-4e90-95d4-bb3e28fd91a4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:40.277350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95985f7d-acad-4e90-95d4-bb3e28fd91a4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:40.277429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95985f7d-acad-4e90-95d4-bb3e28fd91a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:40.295097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-95985f7d-acad-4e90-95d4-bb3e28fd91a4 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:40.298733 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-95985f7d-acad-4e90-95d4-bb3e28fd91a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:40.368256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 421/834] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:40.392468 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918527c-822a-4132-871a-98451edff0ec None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:40.425919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:40.508155 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918527c-822a-4132-871a-98451edff0ec None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:40.545143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589f1af-c0b9-4a08-a4ee-c155687f9c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:40.546215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589f1af-c0b9-4a08-a4ee-c155687f9c7a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:40.546652 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589f1af-c0b9-4a08-a4ee-c155687f9c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:40.547076 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589f1af-c0b9-4a08-a4ee-c155687f9c7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:40.558230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0589f1af-c0b9-4a08-a4ee-c155687f9c7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:40.566504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-0589f1af-c0b9-4a08-a4ee-c155687f9c7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:40.566504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:40.566504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:40.566504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:40.566504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:40.566504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:40.566504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:40.566504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:40.566504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:40.566504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:40.566504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:40.566504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:40.566504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:40.566504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:40.566504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:40.566504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:40.567634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:40.567634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:40.567634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:40.567634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:40.567634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:40.567634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:40.567634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:40.567634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:40.567634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:40.567634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:40.567634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:40.567634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:40.567634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:40.567634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:40.567634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:40.567634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:40.568681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:40.568681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:40.568681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:40.568681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:40.568681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:40.568681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:40.568681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:40.568681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:40.568681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:40.568681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:40.568681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:20:40.568681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:40.570968 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 414/835] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:20:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:40.578316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95985f7d-acad-4e90-95d4-bb3e28fd91a4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:40.687162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95985f7d-acad-4e90-95d4-bb3e28fd91a4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:40.688695 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:40.725628 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bf291-e05b-4b75-8b4f-da7e817aa0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:40.726315 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bf291-e05b-4b75-8b4f-da7e817aa0aa None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:40.726602 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bf291-e05b-4b75-8b4f-da7e817aa0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:40.726883 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bf291-e05b-4b75-8b4f-da7e817aa0aa None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:40.727633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5bf291-e05b-4b75-8b4f-da7e817aa0aa None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:40.728053 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5bf291-e05b-4b75-8b4f-da7e817aa0aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:40.729461 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5bf291-e05b-4b75-8b4f-da7e817aa0aa None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:40.738554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 422/836] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:20:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:40.744017 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589f1af-c0b9-4a08-a4ee-c155687f9c7a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:40.781712 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777ae30-ee15-4a52-b261-57598ce10a9d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:40.782020 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777ae30-ee15-4a52-b261-57598ce10a9d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:40.782204 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777ae30-ee15-4a52-b261-57598ce10a9d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:40.783708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 423/837] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:20:40.788846 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b500a3-3d41-48f1-bdbb-3e7778eb4549 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:40.788846 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b500a3-3d41-48f1-bdbb-3e7778eb4549 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:40.789176 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b500a3-3d41-48f1-bdbb-3e7778eb4549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:40.806719 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-91b500a3-3d41-48f1-bdbb-3e7778eb4549 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:40.808052 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-91b500a3-3d41-48f1-bdbb-3e7778eb4549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:40.848518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:40.860858 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589f1af-c0b9-4a08-a4ee-c155687f9c7a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:40.898651 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffa9a3-adde-484a-add3-e8e8af87d857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:40.899536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffa9a3-adde-484a-add3-e8e8af87d857 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:40.905208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffa9a3-adde-484a-add3-e8e8af87d857 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:40.905208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffa9a3-adde-484a-add3-e8e8af87d857 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:40.905741 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 424/838] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:20:40.914474 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffa9a3-adde-484a-add3-e8e8af87d857 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:40.916061 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffa9a3-adde-484a-add3-e8e8af87d857 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:40.926919 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b500a3-3d41-48f1-bdbb-3e7778eb4549 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:40.933955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-a5ffa9a3-adde-484a-add3-e8e8af87d857 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:40.933955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:40.933955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:40.933955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:40.933955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:40.933955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:40.933955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:40.933955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:40.933955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:40.933955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:40.933955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:40.933955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:40.933955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:40.933955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:40.933955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:40.934879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:40.934879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:40.934879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:40.934879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:40.934879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:40.934879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:40.934879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:40.934879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:40.934879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:40.934879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:40.934879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:40.934879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:40.934879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:40.934879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:40.934879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:40.934879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:40.934879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:40.935755 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:40.935755 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:40.935755 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:40.935755 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:40.939813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 415/839] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:40.941149 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffa9a3-adde-484a-add3-e8e8af87d857 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:41.049173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffa9a3-adde-484a-add3-e8e8af87d857 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:41.056551 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:41.066607 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b500a3-3d41-48f1-bdbb-3e7778eb4549 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:41.086898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70dff0c-b235-4da8-bc6f-a8708a69dec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:41.087783 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70dff0c-b235-4da8-bc6f-a8708a69dec8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:41.088138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70dff0c-b235-4da8-bc6f-a8708a69dec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:41.088493 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70dff0c-b235-4da8-bc6f-a8708a69dec8 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:41.089582 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70dff0c-b235-4da8-bc6f-a8708a69dec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:41.090265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70dff0c-b235-4da8-bc6f-a8708a69dec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:41.091750 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70dff0c-b235-4da8-bc6f-a8708a69dec8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:41.109590 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc4b7f-59d1-47f7-8e82-2447b538d5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:41.110772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc4b7f-59d1-47f7-8e82-2447b538d5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:41.111255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc4b7f-59d1-47f7-8e82-2447b538d5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:41.111742 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc4b7f-59d1-47f7-8e82-2447b538d5f3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:41.111963 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 416/840] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:41.112863 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc4b7f-59d1-47f7-8e82-2447b538d5f3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:41.113327 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70dff0c-b235-4da8-bc6f-a8708a69dec8 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:41.114818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc4b7f-59d1-47f7-8e82-2447b538d5f3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:41.116437 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:41.132784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-13fc4b7f-59d1-47f7-8e82-2447b538d5f3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 08:20:41.132784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:41.132784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:20:41.132784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:20:41.132784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:20:41.132784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:20:41.132784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:20:41.132784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:41.132784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 08:20:41.132784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:41.132784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:41.132784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:41.132784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:41.132784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:41.132784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:41.132784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:41.135633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:41.135633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:41.135633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:41.135633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:41.135633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:41.135633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:41.135633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:41.135633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:20:41.135633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:20:41.135633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:20:41.135633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:20:41.135633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:41.135633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:41.135633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:41.135633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:41.135633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:41.136488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:41.136488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:41.136488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:41.136488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:41.136488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:41.136488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:41.136488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:41.136488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:41.136488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:41.136488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:41.136488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:41.136488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:20:41.136488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:41.136488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 08:20:41.136488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:41.143649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 425/841] 10.222.0.17 () {56 vars in 1132 bytes} [Tue Jun 7 08:20:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:41.144704 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc4b7f-59d1-47f7-8e82-2447b538d5f3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:41.231621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70dff0c-b235-4da8-bc6f-a8708a69dec8 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:41.258968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc4b7f-59d1-47f7-8e82-2447b538d5f3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:41.268173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f04694-7580-47fb-af11-90786d5f5319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:41.268987 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f04694-7580-47fb-af11-90786d5f5319 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:41.269570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f04694-7580-47fb-af11-90786d5f5319 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:41.269570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f04694-7580-47fb-af11-90786d5f5319 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:41.276841 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f04694-7580-47fb-af11-90786d5f5319 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:41.278358 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f04694-7580-47fb-af11-90786d5f5319 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:41.294320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-b0f04694-7580-47fb-af11-90786d5f5319 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:20:41.294320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:41.294320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:41.294320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:41.294320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:41.294320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:41.294320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:41.294320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:41.294320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:41.294320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:41.294320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:41.294320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:41.294320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:41.294320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:41.294320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:41.294320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:41.295469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:41.295469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:41.295469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:41.295469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:41.295469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:41.295469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:41.295469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:41.295469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:41.295469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:41.295469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:41.295469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:41.295469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:41.295469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:41.295469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:41.295469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:41.295469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:41.295469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:41.295469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:41.296535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:41.296535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:41.296535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:41.296535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:41.296535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:41.296535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:41.296535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:41.296535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:41.296535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:20:41.296535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:41.296535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 417/842] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jun 7 08:20:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:41.297165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33a5c9-4d5d-4694-a9ba-36ab1d25ee1b None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:41.297165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33a5c9-4d5d-4694-a9ba-36ab1d25ee1b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:41.297299 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33a5c9-4d5d-4694-a9ba-36ab1d25ee1b None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:41.299011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 418/843] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:41.299502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd39e3f-352f-4451-ba65-cae9581fe744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:41.300275 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd39e3f-352f-4451-ba65-cae9581fe744 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:41.300553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd39e3f-352f-4451-ba65-cae9581fe744 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:41.300836 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd39e3f-352f-4451-ba65-cae9581fe744 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:41.301554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33a5c9-4d5d-4694-a9ba-36ab1d25ee1b None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:41.301693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd39e3f-352f-4451-ba65-cae9581fe744 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:41.302012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd39e3f-352f-4451-ba65-cae9581fe744 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:41.303515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd39e3f-352f-4451-ba65-cae9581fe744 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:41.309205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 426/844] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jun 7 08:20:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:41.311133 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88780008-41e1-4243-9501-4271e5e31b70 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:41.311568 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88780008-41e1-4243-9501-4271e5e31b70 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:41.311995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88780008-41e1-4243-9501-4271e5e31b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:41.330727 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-88780008-41e1-4243-9501-4271e5e31b70 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:41.332067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-88780008-41e1-4243-9501-4271e5e31b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:41.349684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:41.370510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:41.402921 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 427/845] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:41.404201 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88780008-41e1-4243-9501-4271e5e31b70 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:41.425461 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33a5c9-4d5d-4694-a9ba-36ab1d25ee1b None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:41.463336 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef1464-4a16-4c80-aec0-e98039d0f601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:41.464006 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef1464-4a16-4c80-aec0-e98039d0f601 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:41.464292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef1464-4a16-4c80-aec0-e98039d0f601 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:41.464567 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef1464-4a16-4c80-aec0-e98039d0f601 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:41.465434 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef1464-4a16-4c80-aec0-e98039d0f601 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:41.465739 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef1464-4a16-4c80-aec0-e98039d0f601 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:41.467195 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef1464-4a16-4c80-aec0-e98039d0f601 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:41.485278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 419/846] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Jun 7 08:20:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:41.485899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef1464-4a16-4c80-aec0-e98039d0f601 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:41.525496 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88780008-41e1-4243-9501-4271e5e31b70 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:41.562246 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369721a2-5fb4-40b8-a248-6805d0bf0598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:41.562834 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369721a2-5fb4-40b8-a248-6805d0bf0598 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:41.563117 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369721a2-5fb4-40b8-a248-6805d0bf0598 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:41.563397 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369721a2-5fb4-40b8-a248-6805d0bf0598 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:41.564103 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369721a2-5fb4-40b8-a248-6805d0bf0598 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:41.564402 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369721a2-5fb4-40b8-a248-6805d0bf0598 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:41.565810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369721a2-5fb4-40b8-a248-6805d0bf0598 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:41.570179 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 428/847] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jun 7 08:20:41 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:41.570956 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369721a2-5fb4-40b8-a248-6805d0bf0598 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:41.598096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:41.612530 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef1464-4a16-4c80-aec0-e98039d0f601 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:41.642868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:41.672893 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d798a-e919-4be9-859a-3751f86facef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:41.674603 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d798a-e919-4be9-859a-3751f86facef None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:41.674954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d798a-e919-4be9-859a-3751f86facef None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:41.675556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d798a-e919-4be9-859a-3751f86facef None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:41.676867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d798a-e919-4be9-859a-3751f86facef None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:41.678671 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d798a-e919-4be9-859a-3751f86facef None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:41.696150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-c17d798a-e919-4be9-859a-3751f86facef None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 08:20:41.696150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:41.696150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:20:41.696150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:20:41.696150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:20:41.696150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:20:41.696150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:20:41.696150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:41.696150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 08:20:41.696150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:41.696150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:41.696150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:41.696150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:41.696150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:41.696150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:41.696150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:41.696150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:41.697260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:41.697260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:41.697260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:41.697260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:41.697260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:41.697260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:41.697260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:20:41.697260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:20:41.697260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:20:41.697260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:20:41.697260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:41.697260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:41.697260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:41.697260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:41.697260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:41.697260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:41.698093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:41.698093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:41.698093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:41.698093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:41.698093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:41.698093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:41.698093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:41.698093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:41.698093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:41.698093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:41.698093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:20:41.698093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:41.698093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 08:20:41.698093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:41.699400 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369721a2-5fb4-40b8-a248-6805d0bf0598 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:41.701167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 420/848] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:20:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:41.704633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d798a-e919-4be9-859a-3751f86facef None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:41.745432 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e99f4-641a-44e3-bafb-253d1b13f42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:41.746561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e99f4-641a-44e3-bafb-253d1b13f42b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:41.746848 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e99f4-641a-44e3-bafb-253d1b13f42b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:41.747683 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e99f4-641a-44e3-bafb-253d1b13f42b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:41.834369 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10e99f4-641a-44e3-bafb-253d1b13f42b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:41.836203 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10e99f4-641a-44e3-bafb-253d1b13f42b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:41.842536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d798a-e919-4be9-859a-3751f86facef None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:41.859105 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:41.877917 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3d302-89f8-4fb8-a51b-ce3082b8b786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:41.878585 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3d302-89f8-4fb8-a51b-ce3082b8b786 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:41.878868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3d302-89f8-4fb8-a51b-ce3082b8b786 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:41.879145 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3d302-89f8-4fb8-a51b-ce3082b8b786 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:41.879915 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3d302-89f8-4fb8-a51b-ce3082b8b786 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:41.880225 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3d302-89f8-4fb8-a51b-ce3082b8b786 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:41.881657 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3d302-89f8-4fb8-a51b-ce3082b8b786 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:41.886368 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 421/849] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:20:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:41.887594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454daaa6-4fe3-4366-a66e-4af67d0b27a0 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:41.887908 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454daaa6-4fe3-4366-a66e-4af67d0b27a0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:41.888191 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454daaa6-4fe3-4366-a66e-4af67d0b27a0 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:41.889703 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 422/850] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:41.894395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454daaa6-4fe3-4366-a66e-4af67d0b27a0 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:41.945423 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:41.961925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 429/851] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:41 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:20:41.961925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcc40d-14cc-4827-897b-d9064276ecb4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:41.961925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcc40d-14cc-4827-897b-d9064276ecb4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:41.961925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcc40d-14cc-4827-897b-d9064276ecb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:41.978937 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a0bcc40d-14cc-4827-897b-d9064276ecb4 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:41.980126 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a0bcc40d-14cc-4827-897b-d9064276ecb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:41.999145 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454daaa6-4fe3-4366-a66e-4af67d0b27a0 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:42.033502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354c352-70df-4172-9e56-ed23614364aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:42.034105 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354c352-70df-4172-9e56-ed23614364aa None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:42.034381 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354c352-70df-4172-9e56-ed23614364aa None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:42.034699 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354c352-70df-4172-9e56-ed23614364aa None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:42.035397 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354c352-70df-4172-9e56-ed23614364aa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:42.035699 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354c352-70df-4172-9e56-ed23614364aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:42.037075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354c352-70df-4172-9e56-ed23614364aa None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:42.042107 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 423/852] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:20:41 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:42.042708 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354c352-70df-4172-9e56-ed23614364aa None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:42.062305 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 430/853] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:42.085624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bcc40d-14cc-4827-897b-d9064276ecb4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:42.155752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:42.163810 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354c352-70df-4172-9e56-ed23614364aa None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:42.181615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:42.209603 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16873429-22c9-453c-8bb6-31191c17b3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:42.210932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16873429-22c9-453c-8bb6-31191c17b3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:42.210932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16873429-22c9-453c-8bb6-31191c17b3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:42.211278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16873429-22c9-453c-8bb6-31191c17b3b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:42.226778 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16873429-22c9-453c-8bb6-31191c17b3b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:42.228774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16873429-22c9-453c-8bb6-31191c17b3b2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:42.235143 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bcc40d-14cc-4827-897b-d9064276ecb4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:42.251329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-16873429-22c9-453c-8bb6-31191c17b3b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:42.251329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:42.251329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:42.251329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:42.251329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:42.251329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:42.251329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:42.251329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:42.251329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:42.251329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:42.251329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:42.251329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:42.251329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:42.251329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:42.251329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:42.252277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:42.252277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:42.252277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:42.252277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:42.252277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:42.252277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:42.252277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:42.252277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:42.252277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:42.252277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:42.252277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:42.252277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:42.252277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:42.252277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:42.252277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:42.252277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:42.252277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:42.258786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:42.258786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:42.258786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:42.258786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:42.258786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 424/854] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:42.263414 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16873429-22c9-453c-8bb6-31191c17b3b2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:42.275766 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2520d5-aee3-4557-ac45-f2228adec8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:42.276562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2520d5-aee3-4557-ac45-f2228adec8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:42.276837 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2520d5-aee3-4557-ac45-f2228adec8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:42.277137 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2520d5-aee3-4557-ac45-f2228adec8a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:42.288142 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2520d5-aee3-4557-ac45-f2228adec8a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:42.295949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5e2520d5-aee3-4557-ac45-f2228adec8a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:42.295949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:42.295949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:42.295949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:42.295949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:42.295949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:42.295949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:42.295949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:42.295949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:42.295949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:42.295949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:42.295949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:42.295949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:42.295949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:42.295949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:42.295949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:42.296886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:42.296886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:42.296886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:42.296886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:42.296886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:42.296886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:42.296886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:42.296886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:42.296886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:42.296886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:42.296886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:42.296886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:42.296886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:42.296886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:42.296886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:42.296886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:42.297737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:42.297737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:42.297737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:42.297737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:42.297737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:42.297737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:42.297737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:42.297737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:42.297737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:42.297737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:42.297737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:20:42.297737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:42.297737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 431/855] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:20:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:42.304884 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2520d5-aee3-4557-ac45-f2228adec8a6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:42.414472 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2520d5-aee3-4557-ac45-f2228adec8a6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:42.418217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16873429-22c9-453c-8bb6-31191c17b3b2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:42.423992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:42.452244 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc424ad6-a5e1-4c22-87f5-9c99a54390c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:42.452911 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc424ad6-a5e1-4c22-87f5-9c99a54390c2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:42.453218 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc424ad6-a5e1-4c22-87f5-9c99a54390c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:42.453491 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc424ad6-a5e1-4c22-87f5-9c99a54390c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:42.454240 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc424ad6-a5e1-4c22-87f5-9c99a54390c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:42.454712 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc424ad6-a5e1-4c22-87f5-9c99a54390c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:42.456151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc424ad6-a5e1-4c22-87f5-9c99a54390c2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:42.470262 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 432/856] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:42.470440 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ef660-e1a4-4249-ae50-dcd1e7bff651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:42.470440 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ef660-e1a4-4249-ae50-dcd1e7bff651 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:42.470726 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ef660-e1a4-4249-ae50-dcd1e7bff651 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:42.470957 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ef660-e1a4-4249-ae50-dcd1e7bff651 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:42.471687 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ef660-e1a4-4249-ae50-dcd1e7bff651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:42.471989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ef660-e1a4-4249-ae50-dcd1e7bff651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:42.472399 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc424ad6-a5e1-4c22-87f5-9c99a54390c2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:42.473484 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ef660-e1a4-4249-ae50-dcd1e7bff651 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:42.486767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 425/857] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:42.492540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ef660-e1a4-4249-ae50-dcd1e7bff651 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:42.520225 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:42.601162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc424ad6-a5e1-4c22-87f5-9c99a54390c2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:42.612273 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ef660-e1a4-4249-ae50-dcd1e7bff651 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:42.636981 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a89eb-c795-4215-bb20-0ba94ec13ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:42.638659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a89eb-c795-4215-bb20-0ba94ec13ebe None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:42.639121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a89eb-c795-4215-bb20-0ba94ec13ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:42.639571 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a89eb-c795-4215-bb20-0ba94ec13ebe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:42.650612 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a89eb-c795-4215-bb20-0ba94ec13ebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:42.652339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a89eb-c795-4215-bb20-0ba94ec13ebe None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:42.656821 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9248b3-2227-438a-b68d-d50bb1fe32f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:42.657639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9248b3-2227-438a-b68d-d50bb1fe32f9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:42.657916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9248b3-2227-438a-b68d-d50bb1fe32f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:42.658189 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9248b3-2227-438a-b68d-d50bb1fe32f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:42.666540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9248b3-2227-438a-b68d-d50bb1fe32f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:42.667982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9248b3-2227-438a-b68d-d50bb1fe32f9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:42.671467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-2d1a89eb-c795-4215-bb20-0ba94ec13ebe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:42.671467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:42.671467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:42.671467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:42.671467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:42.671467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:42.671467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:42.671467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:42.671467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:42.671467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:42.671467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:42.671467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:42.671467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:42.671467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:42.671467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:42.672320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:42.672320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:42.672320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:42.672320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:42.672320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:42.672320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:42.672320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:42.672320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:42.672320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:42.672320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:42.672320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:42.672320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:42.672320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:42.672320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:42.672320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:42.672320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:42.672320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:42.673221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:42.673221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:42.673221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:42.673221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:42.675643 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 433/858] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:42.685092 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a89eb-c795-4215-bb20-0ba94ec13ebe None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:42.685229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-1a9248b3-2227-438a-b68d-d50bb1fe32f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:42.685229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:42.685229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:42.685229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:42.685229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:42.685229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:42.685229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:42.685229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:42.685229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:42.685229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:42.685229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:42.685229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:42.685229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:42.685229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:42.685229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:42.686033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:42.686033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:42.686033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:42.686033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:42.686033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:42.686033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:42.686033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:42.686033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:42.686033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:42.686033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:42.686033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:42.686033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:42.686033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:42.686033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:42.686033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:42.686033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:42.686033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:42.686923 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:42.686923 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:42.686923 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:42.686923 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:42.686923 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:42.686923 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:42.686923 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:42.686923 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:42.686923 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 426/859] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:42.692707 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9248b3-2227-438a-b68d-d50bb1fe32f9 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:42.712134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:42.790760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:42.812044 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9248b3-2227-438a-b68d-d50bb1fe32f9 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:42.818913 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a89eb-c795-4215-bb20-0ba94ec13ebe None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:42.859143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c954601-0924-44b7-8d29-8d7c9c255c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:42.859807 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c954601-0924-44b7-8d29-8d7c9c255c31 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:42.860080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c954601-0924-44b7-8d29-8d7c9c255c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:42.860356 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c954601-0924-44b7-8d29-8d7c9c255c31 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:42.861171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c954601-0924-44b7-8d29-8d7c9c255c31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:42.861477 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c954601-0924-44b7-8d29-8d7c9c255c31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:42.862073 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365be3a-ffa2-4b97-93eb-ef68ff3dcb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:42.862957 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365be3a-ffa2-4b97-93eb-ef68ff3dcb1a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:42.863075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c954601-0924-44b7-8d29-8d7c9c255c31 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:42.863540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365be3a-ffa2-4b97-93eb-ef68ff3dcb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:42.864024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365be3a-ffa2-4b97-93eb-ef68ff3dcb1a None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:42.864937 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9365be3a-ffa2-4b97-93eb-ef68ff3dcb1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:42.865476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9365be3a-ffa2-4b97-93eb-ef68ff3dcb1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:42.867042 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9365be3a-ffa2-4b97-93eb-ef68ff3dcb1a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:42.881035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 427/860] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:42.884620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c954601-0924-44b7-8d29-8d7c9c255c31 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:42.885477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 434/861] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:42.890339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365be3a-ffa2-4b97-93eb-ef68ff3dcb1a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:42.998219 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:43.000046 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c954601-0924-44b7-8d29-8d7c9c255c31 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:43.016680 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365be3a-ffa2-4b97-93eb-ef68ff3dcb1a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:43.043318 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a6f9c-0866-4277-8dda-01643ed8638d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:43.044875 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a6f9c-0866-4277-8dda-01643ed8638d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:43.045331 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a6f9c-0866-4277-8dda-01643ed8638d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:43.046070 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a6f9c-0866-4277-8dda-01643ed8638d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:43.047327 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a6f9c-0866-4277-8dda-01643ed8638d None admin] RBAC: Adding query filter params (role.id=a04f2ab5f1234bd6b1dc4c18b4210a89, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=bbf0ce396885472bb5d225aa009043f2) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:43.047793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a6f9c-0866-4277-8dda-01643ed8638d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:43.050329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a6f9c-0866-4277-8dda-01643ed8638d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:43.053808 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86bb48-881b-4eaf-bf9c-7c5aa01cff9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:43.055020 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86bb48-881b-4eaf-bf9c-7c5aa01cff9d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:43.055538 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86bb48-881b-4eaf-bf9c-7c5aa01cff9d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:43.056076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86bb48-881b-4eaf-bf9c-7c5aa01cff9d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:43.057782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 428/862] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:42 2022] GET /identity/v3/role_assignments?user.id=bbf0ce396885472bb5d225aa009043f2&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:43.069033 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86bb48-881b-4eaf-bf9c-7c5aa01cff9d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:43.069033 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86bb48-881b-4eaf-bf9c-7c5aa01cff9d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:43.086415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-eb86bb48-881b-4eaf-bf9c-7c5aa01cff9d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:20:43.086415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:43.086415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:43.086415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:43.086415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:43.086415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:43.086415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:43.086415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:43.086415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:43.086415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:43.086415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:43.086415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:43.086415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:43.086415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:43.086415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:43.086415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:43.087448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:43.087448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:43.087448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:43.087448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:43.087448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:43.087448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:43.087448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:43.087448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:43.087448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:43.087448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:43.087448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:43.087448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:43.087448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:43.087448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:43.087448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:43.087448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:43.087448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:43.087448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:43.088382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:43.088382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:43.088382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:43.088382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:43.088382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:43.088382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:43.088382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:43.088382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:43.088382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 08:20:43.088382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:43.093765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 435/863] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:20:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:43.095179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a6f9c-0866-4277-8dda-01643ed8638d None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:43.113109 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:43.223193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a6f9c-0866-4277-8dda-01643ed8638d None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:43.264297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757914e0-f61b-42cf-93af-04d1295a025f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:43.265413 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757914e0-f61b-42cf-93af-04d1295a025f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:43.265924 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757914e0-f61b-42cf-93af-04d1295a025f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:43.266196 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757914e0-f61b-42cf-93af-04d1295a025f None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:43.267032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757914e0-f61b-42cf-93af-04d1295a025f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:43.267335 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757914e0-f61b-42cf-93af-04d1295a025f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:43.268879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757914e0-f61b-42cf-93af-04d1295a025f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:43.288072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 429/864] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Jun 7 08:20:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:43.293556 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86bb48-881b-4eaf-bf9c-7c5aa01cff9d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:43.396012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86bb48-881b-4eaf-bf9c-7c5aa01cff9d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:43.431459 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d319d-a43c-417e-9d49-6e9754f6684e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:43.432110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d319d-a43c-417e-9d49-6e9754f6684e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:43.432385 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d319d-a43c-417e-9d49-6e9754f6684e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:43.432656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d319d-a43c-417e-9d49-6e9754f6684e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:43.443363 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:43.509607 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d319d-a43c-417e-9d49-6e9754f6684e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:43.511061 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d319d-a43c-417e-9d49-6e9754f6684e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:43.619447 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 436/865] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:43 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:20:43.624555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757914e0-f61b-42cf-93af-04d1295a025f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:43.691499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:43.727444 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757914e0-f61b-42cf-93af-04d1295a025f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:43.766320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b806b-aea8-48b0-ba31-846a295a5858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:43.767442 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b806b-aea8-48b0-ba31-846a295a5858 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:43.767908 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b806b-aea8-48b0-ba31-846a295a5858 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:43.768355 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b806b-aea8-48b0-ba31-846a295a5858 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:43.779474 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b806b-aea8-48b0-ba31-846a295a5858 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:43.781185 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b806b-aea8-48b0-ba31-846a295a5858 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:43.799023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-355b806b-aea8-48b0-ba31-846a295a5858 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:43.799023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:43.799023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:43.799023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:43.799023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:43.799023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:43.799023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:43.799023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:43.799023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:43.799023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:43.799023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:43.799023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:43.799023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:43.799023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:43.799023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:43.799933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:43.799933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:43.799933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:43.799933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:43.799933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:43.799933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:43.799933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:43.799933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:43.799933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:43.799933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:43.799933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:43.799933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:43.799933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:43.799933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:43.799933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:43.799933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:43.799933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:43.800802 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:43.800802 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:43.800802 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:43.800802 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:43.802764 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 430/866] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:43.809956 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d319d-a43c-417e-9d49-6e9754f6684e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:43.849775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:43.915690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d319d-a43c-417e-9d49-6e9754f6684e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:43.933534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711f0a3-7a98-4b70-9723-a01a790d7957 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:43.934153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711f0a3-7a98-4b70-9723-a01a790d7957 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:43.934686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711f0a3-7a98-4b70-9723-a01a790d7957 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:43.936563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 431/867] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:20:43.941784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36a7f9-c9ee-43e0-8778-bc8b843871d3 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:43.942171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36a7f9-c9ee-43e0-8778-bc8b843871d3 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:43.942454 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36a7f9-c9ee-43e0-8778-bc8b843871d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:43.951969 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1a806-f513-4eaf-9c9e-2157edf44f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:43.952639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1a806-f513-4eaf-9c9e-2157edf44f6d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:43.952924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1a806-f513-4eaf-9c9e-2157edf44f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:43.953230 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1a806-f513-4eaf-9c9e-2157edf44f6d None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:43.953957 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1a806-f513-4eaf-9c9e-2157edf44f6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:43.954731 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1a806-f513-4eaf-9c9e-2157edf44f6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:43.956307 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1a806-f513-4eaf-9c9e-2157edf44f6d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:43.960395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ae36a7f9-c9ee-43e0-8778-bc8b843871d3 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:43.961633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ae36a7f9-c9ee-43e0-8778-bc8b843871d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:43.972858 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 437/868] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:43.978384 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1a806-f513-4eaf-9c9e-2157edf44f6d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:44.037286 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 432/869] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:20:44.055271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36a7f9-c9ee-43e0-8778-bc8b843871d3 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:44.097835 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:44.102652 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1a806-f513-4eaf-9c9e-2157edf44f6d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:44.145499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:44.153595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f7734-17f6-4b75-8c7d-556a5bf8f208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:44.154604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f7734-17f6-4b75-8c7d-556a5bf8f208 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:44.154940 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f7734-17f6-4b75-8c7d-556a5bf8f208 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:44.155222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f7734-17f6-4b75-8c7d-556a5bf8f208 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:44.162419 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f7734-17f6-4b75-8c7d-556a5bf8f208 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:44.164039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f7734-17f6-4b75-8c7d-556a5bf8f208 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:44.167079 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36a7f9-c9ee-43e0-8778-bc8b843871d3 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:44.178390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-765f7734-17f6-4b75-8c7d-556a5bf8f208 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:44.178390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:44.178390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:44.178390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:44.178390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:44.178390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:44.178390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:44.178390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:44.178390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:44.178390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:44.178390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:44.178390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:44.178390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:44.178390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:44.178390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:44.179291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:44.179291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:44.179291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:44.179291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:44.179291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:44.179291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:44.179291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:44.179291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:44.179291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:44.179291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:44.179291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:44.179291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:44.179291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:44.179291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:44.179291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:44.179291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:44.179291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:44.180588 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:44.180588 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:44.180588 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:44.180588 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:44.180588 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:44.180588 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:44.180588 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:44.180588 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:44.180588 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 438/870] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:44.188773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f7734-17f6-4b75-8c7d-556a5bf8f208 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:44.216224 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933dd671-0399-4225-9bce-d9af89f6bb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:44.217450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dd671-0399-4225-9bce-d9af89f6bb29 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:44.217913 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dd671-0399-4225-9bce-d9af89f6bb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:44.218585 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dd671-0399-4225-9bce-d9af89f6bb29 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:44.219891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933dd671-0399-4225-9bce-d9af89f6bb29 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:44.221997 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933dd671-0399-4225-9bce-d9af89f6bb29 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:44.293396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f7734-17f6-4b75-8c7d-556a5bf8f208 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:44.337331 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 433/871] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jun 7 08:20:44 2022] POST /identity/v3/services => generated 269 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:44.340689 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f6a45-e818-426b-af0a-b7401b9f4880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:44.341507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f6a45-e818-426b-af0a-b7401b9f4880 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:44.341793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f6a45-e818-426b-af0a-b7401b9f4880 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:44.342124 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f6a45-e818-426b-af0a-b7401b9f4880 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:44.343025 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f6a45-e818-426b-af0a-b7401b9f4880 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:44.343393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f6a45-e818-426b-af0a-b7401b9f4880 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:44.345030 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f6a45-e818-426b-af0a-b7401b9f4880 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:44.360510 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 439/872] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:44.366576 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933dd671-0399-4225-9bce-d9af89f6bb29 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:44.486257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933dd671-0399-4225-9bce-d9af89f6bb29 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:44.489651 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3f956-5743-479e-bf99-d1ea6734bb60 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:44.490150 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3f956-5743-479e-bf99-d1ea6734bb60 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:44.490226 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3f956-5743-479e-bf99-d1ea6734bb60 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:44.491828 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 440/873] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:44.497074 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:44.498354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fede2-c9a0-46c4-9558-04ee466d63ae None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:44.498638 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fede2-c9a0-46c4-9558-04ee466d63ae None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:44.498916 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fede2-c9a0-46c4-9558-04ee466d63ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:44.520112 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ab4fede2-c9a0-46c4-9558-04ee466d63ae None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:44.521819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ab4fede2-c9a0-46c4-9558-04ee466d63ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:44.524790 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79fa02-cbe4-4ca1-8651-e4b896368891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:44.525659 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79fa02-cbe4-4ca1-8651-e4b896368891 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:44.525947 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79fa02-cbe4-4ca1-8651-e4b896368891 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:44.526226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79fa02-cbe4-4ca1-8651-e4b896368891 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:44.527126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79fa02-cbe4-4ca1-8651-e4b896368891 None admin] RBAC: Adding query filter params (role.id=8c63c00418a346b29f7db3f87a404e70, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=334dc47ae2e643ccb44d25c6b5906f14) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:44.527498 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79fa02-cbe4-4ca1-8651-e4b896368891 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:44.529049 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79fa02-cbe4-4ca1-8651-e4b896368891 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:44.541080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 434/874] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:44 2022] GET /identity/v3/role_assignments?user.id=334dc47ae2e643ccb44d25c6b5906f14&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:44.541285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:44.604292 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 441/875] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:44 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:20:44.628103 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79fa02-cbe4-4ca1-8651-e4b896368891 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:44.741848 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79fa02-cbe4-4ca1-8651-e4b896368891 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:44.780464 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708fc51-0905-4b23-a897-311d1862a6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:44.783402 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708fc51-0905-4b23-a897-311d1862a6f9 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:44.783402 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708fc51-0905-4b23-a897-311d1862a6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:44.783402 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708fc51-0905-4b23-a897-311d1862a6f9 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:44.783402 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708fc51-0905-4b23-a897-311d1862a6f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:44.783843 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708fc51-0905-4b23-a897-311d1862a6f9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:44.957826 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 435/876] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jun 7 08:20:44 2022] POST /identity/v3/services => generated 251 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:45.485109 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25058a-5e84-4d57-a82a-d03864cb51f3 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:45.485818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25058a-5e84-4d57-a82a-d03864cb51f3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:45.486319 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25058a-5e84-4d57-a82a-d03864cb51f3 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:45.488156 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 442/877] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:45.493695 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5c356-4984-4cad-ac5a-fcd08392f5d5 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:45.494226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5c356-4984-4cad-ac5a-fcd08392f5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:45.494723 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5c356-4984-4cad-ac5a-fcd08392f5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:45.512334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-10f5c356-4984-4cad-ac5a-fcd08392f5d5 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:45.514438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-10f5c356-4984-4cad-ac5a-fcd08392f5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:45.541764 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:45.598335 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 436/878] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:45.621701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25058a-5e84-4d57-a82a-d03864cb51f3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:45.727264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25058a-5e84-4d57-a82a-d03864cb51f3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:45.767288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16247914-1493-46fb-9baf-465e48393e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:45.768032 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16247914-1493-46fb-9baf-465e48393e36 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:45.768317 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16247914-1493-46fb-9baf-465e48393e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:45.768595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16247914-1493-46fb-9baf-465e48393e36 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:45.779918 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16247914-1493-46fb-9baf-465e48393e36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:45.788405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-16247914-1493-46fb-9baf-465e48393e36 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:20:45.788405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:45.788405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:45.788405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:45.788405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:45.788405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:45.788405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:45.788405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:45.788405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:45.788405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:45.788405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:45.788405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:45.788405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:45.788405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:45.788405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:45.788405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:45.789509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:45.789509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:45.789509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:45.789509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:45.789509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:45.789509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:45.789509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:45.789509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:45.789509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:45.789509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:45.789509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:45.789509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:45.789509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:45.789509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:45.789509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:45.789509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:45.790396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:45.790396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:45.790396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:45.790396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:45.790396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:45.790396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:45.790396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:45.790396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:45.790396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:45.790396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:45.790396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:20:45.790396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:45.790396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 443/879] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Jun 7 08:20:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:45.797777 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5c356-4984-4cad-ac5a-fcd08392f5d5 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:45.901032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5c356-4984-4cad-ac5a-fcd08392f5d5 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:45.935725 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae5b32-a696-4015-a31d-707dac771073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:45.936397 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae5b32-a696-4015-a31d-707dac771073 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:45.936682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae5b32-a696-4015-a31d-707dac771073 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:45.936959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae5b32-a696-4015-a31d-707dac771073 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:45.937866 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae5b32-a696-4015-a31d-707dac771073 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:45.938180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae5b32-a696-4015-a31d-707dac771073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:45.939557 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae5b32-a696-4015-a31d-707dac771073 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:45.948158 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 437/880] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Jun 7 08:20:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:45.954065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16247914-1493-46fb-9baf-465e48393e36 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:45.964364 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:46.057989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16247914-1493-46fb-9baf-465e48393e36 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:46.093069 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f9e40-8348-4a79-ab29-4b7ed06146f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:46.093902 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f9e40-8348-4a79-ab29-4b7ed06146f7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:46.094185 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f9e40-8348-4a79-ab29-4b7ed06146f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:46.094461 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f9e40-8348-4a79-ab29-4b7ed06146f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:46.106274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f9e40-8348-4a79-ab29-4b7ed06146f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:46.108286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f9e40-8348-4a79-ab29-4b7ed06146f7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:46.128121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-f83f9e40-8348-4a79-ab29-4b7ed06146f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:46.128121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:46.128121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:46.128121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:46.128121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:46.128121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:46.128121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:46.128121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:46.128121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:46.128121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:46.128121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:46.128121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:46.128121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:46.128121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:46.128121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:46.129127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:46.129127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:46.129127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:46.129127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:46.129127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:46.129127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:46.129127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:46.129127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:46.129127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:46.129127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:46.129127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:46.129127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:46.129127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:46.129127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:46.129127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:46.129127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:46.129127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:46.130036 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:46.130036 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:46.130036 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:46.130036 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:46.132309 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 444/881] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:46.137663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae5b32-a696-4015-a31d-707dac771073 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:46.159974 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:46.254763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae5b32-a696-4015-a31d-707dac771073 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:46.289928 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66ffc0-8b1c-4d7a-a88b-a562135fe13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:46.290596 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66ffc0-8b1c-4d7a-a88b-a562135fe13c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:46.290882 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66ffc0-8b1c-4d7a-a88b-a562135fe13c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:46.291161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66ffc0-8b1c-4d7a-a88b-a562135fe13c None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:46.291924 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66ffc0-8b1c-4d7a-a88b-a562135fe13c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:46.292231 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66ffc0-8b1c-4d7a-a88b-a562135fe13c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:46.293646 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66ffc0-8b1c-4d7a-a88b-a562135fe13c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:46.307245 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 438/882] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:46.314132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f9e40-8348-4a79-ab29-4b7ed06146f7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:46.376167 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:46.423937 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f9e40-8348-4a79-ab29-4b7ed06146f7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:46.457616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2240a6-2e9a-4bde-8ed0-96dd77673d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:46.458808 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2240a6-2e9a-4bde-8ed0-96dd77673d48 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:46.459307 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2240a6-2e9a-4bde-8ed0-96dd77673d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:46.459797 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2240a6-2e9a-4bde-8ed0-96dd77673d48 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:46.468335 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2240a6-2e9a-4bde-8ed0-96dd77673d48 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:46.470090 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2240a6-2e9a-4bde-8ed0-96dd77673d48 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:46.485653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-6e2240a6-2e9a-4bde-8ed0-96dd77673d48 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:20:46.485653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:46.485653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:46.485653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:46.485653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:46.485653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:46.485653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:46.485653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:46.485653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:46.485653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:46.485653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:46.485653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:46.485653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:46.485653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:46.485653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:46.485653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:46.486630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:46.486630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:46.486630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:46.486630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:46.486630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:46.486630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:46.486630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:46.486630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:46.486630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:46.486630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:46.486630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:46.486630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:46.486630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:46.486630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:46.486630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:46.486630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:46.486630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:46.486630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:46.487589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:46.487589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:46.487589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:46.487589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:46.487589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:46.487589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:46.487589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:46.487589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:46.487589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:20:46.487589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:46.490235 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 445/883] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jun 7 08:20:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:46.496128 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66ffc0-8b1c-4d7a-a88b-a562135fe13c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:46.582534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:46.602770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66ffc0-8b1c-4d7a-a88b-a562135fe13c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:46.638981 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf6855-bd20-41ea-b113-4c22f4786d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:46.639659 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf6855-bd20-41ea-b113-4c22f4786d3f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:46.639944 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf6855-bd20-41ea-b113-4c22f4786d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:46.640218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf6855-bd20-41ea-b113-4c22f4786d3f None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:46.641091 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf6855-bd20-41ea-b113-4c22f4786d3f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:46.641407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf6855-bd20-41ea-b113-4c22f4786d3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:46.642836 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf6855-bd20-41ea-b113-4c22f4786d3f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:46.661792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 439/884] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Jun 7 08:20:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:46.667083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2240a6-2e9a-4bde-8ed0-96dd77673d48 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:46.734619 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef4a5a-6ba4-434e-aa56-1b83da506165 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:46.734963 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef4a5a-6ba4-434e-aa56-1b83da506165 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:46.735171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef4a5a-6ba4-434e-aa56-1b83da506165 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:46.736650 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 440/885] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:20:46.741812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc4be1-ecc3-49a3-b578-2a731b76e93c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:46.742148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc4be1-ecc3-49a3-b578-2a731b76e93c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:46.742433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc4be1-ecc3-49a3-b578-2a731b76e93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:46.764949 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-29dc4be1-ecc3-49a3-b578-2a731b76e93c None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:46.764949 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-29dc4be1-ecc3-49a3-b578-2a731b76e93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:46.786982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2240a6-2e9a-4bde-8ed0-96dd77673d48 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:46.805659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:46.829026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768c62f-0219-465e-a3f9-a17abf4ddcf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:46.829687 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768c62f-0219-465e-a3f9-a17abf4ddcf7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:46.829973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768c62f-0219-465e-a3f9-a17abf4ddcf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:46.830252 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768c62f-0219-465e-a3f9-a17abf4ddcf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:46.849973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 441/886] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:46.871647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc4be1-ecc3-49a3-b578-2a731b76e93c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:46.908832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768c62f-0219-465e-a3f9-a17abf4ddcf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:46.910402 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768c62f-0219-465e-a3f9-a17abf4ddcf7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:46.954096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:46.992478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc4be1-ecc3-49a3-b578-2a731b76e93c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:47.018407 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 446/887] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:46 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:20:47.023443 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768c62f-0219-465e-a3f9-a17abf4ddcf7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:47.034212 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599986a-5751-40a4-bf64-3dc79290e0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:47.035174 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599986a-5751-40a4-bf64-3dc79290e0ab None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.035548 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599986a-5751-40a4-bf64-3dc79290e0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.036351 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599986a-5751-40a4-bf64-3dc79290e0ab None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.037446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599986a-5751-40a4-bf64-3dc79290e0ab None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:47.039161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599986a-5751-40a4-bf64-3dc79290e0ab None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:47.044227 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:47.056421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-0599986a-5751-40a4-bf64-3dc79290e0ab None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 08:20:47.056421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:47.056421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:20:47.056421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:20:47.056421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:20:47.056421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:20:47.056421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:20:47.056421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:47.056421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 08:20:47.056421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:47.056421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:47.056421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:47.056421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:47.056421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:47.056421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:47.056421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:47.059000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:47.059000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:47.059000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:47.059000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:47.059000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:47.059000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:47.059000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:47.059000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:20:47.059000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:20:47.059000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:20:47.059000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:20:47.059000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:47.059000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:47.059000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:47.059000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:47.059000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:47.060595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:47.060595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:47.060595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:47.060595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:47.060595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:47.060595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:47.060595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:47.060595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:47.060595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:47.060595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:47.060595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:47.060595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:20:47.060595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:47.060595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 08:20:47.060595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:47.060595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 442/888] 10.222.0.17 () {56 vars in 1132 bytes} [Tue Jun 7 08:20:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:47.065016 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66417c7-c09e-4d29-8a8d-e804d55f3a37 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.065304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66417c7-c09e-4d29-8a8d-e804d55f3a37 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.065577 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66417c7-c09e-4d29-8a8d-e804d55f3a37 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.067254 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 443/889] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:47.067716 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66417c7-c09e-4d29-8a8d-e804d55f3a37 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:47.139742 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768c62f-0219-465e-a3f9-a17abf4ddcf7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:47.180884 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b334ff-000d-4548-b262-f369bc290b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:47.181728 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b334ff-000d-4548-b262-f369bc290b1b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.182005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b334ff-000d-4548-b262-f369bc290b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.182274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b334ff-000d-4548-b262-f369bc290b1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.182693 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66417c7-c09e-4d29-8a8d-e804d55f3a37 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:47.199748 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b334ff-000d-4548-b262-f369bc290b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:47.201899 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b334ff-000d-4548-b262-f369bc290b1b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:47.217764 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:47.219347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-47b334ff-000d-4548-b262-f369bc290b1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:47.219347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:47.219347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:47.219347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:47.219347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:47.219347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:47.219347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:47.219347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:47.219347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:47.219347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:47.219347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:47.219347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:47.219347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:47.219347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:47.219347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:47.220245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:47.220245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:47.220245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:47.220245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:47.220245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:47.220245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:47.220245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:47.220245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:47.220245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:47.220245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:47.220245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:47.220245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:47.220245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:47.220245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:47.220245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:47.220245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:47.220245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:47.221765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:47.221765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:47.221765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:47.221765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:47.221765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 447/890] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:47.221765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb30b5-ea7d-41a5-9416-63d94e7da032 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.222116 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb30b5-ea7d-41a5-9416-63d94e7da032 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.222346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb30b5-ea7d-41a5-9416-63d94e7da032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.229095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38374e36-7436-4863-9951-45e250fb8d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:47.235490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38374e36-7436-4863-9951-45e250fb8d97 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.235490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38374e36-7436-4863-9951-45e250fb8d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.235490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38374e36-7436-4863-9951-45e250fb8d97 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.235490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38374e36-7436-4863-9951-45e250fb8d97 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:47.235490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38374e36-7436-4863-9951-45e250fb8d97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:47.235490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38374e36-7436-4863-9951-45e250fb8d97 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:47.238345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-92cb30b5-ea7d-41a5-9416-63d94e7da032 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:47.239518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-92cb30b5-ea7d-41a5-9416-63d94e7da032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:47.243130 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 444/891] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jun 7 08:20:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:47.243984 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38374e36-7436-4863-9951-45e250fb8d97 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:47.309037 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:47.314397 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 448/892] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:47.315202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb30b5-ea7d-41a5-9416-63d94e7da032 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:47.350337 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38374e36-7436-4863-9951-45e250fb8d97 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:47.384242 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc23cd9-a992-4c64-b57f-53ecde215711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:47.384897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc23cd9-a992-4c64-b57f-53ecde215711 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.385211 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc23cd9-a992-4c64-b57f-53ecde215711 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.385489 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc23cd9-a992-4c64-b57f-53ecde215711 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.386213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc23cd9-a992-4c64-b57f-53ecde215711 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:47.386523 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc23cd9-a992-4c64-b57f-53ecde215711 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:47.388174 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc23cd9-a992-4c64-b57f-53ecde215711 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:47.401336 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 445/893] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:47.402648 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650f976-4c69-4447-a892-6e9519eee27a None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.402938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650f976-4c69-4447-a892-6e9519eee27a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.403243 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650f976-4c69-4447-a892-6e9519eee27a None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.404682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 446/894] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:47.405521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650f976-4c69-4447-a892-6e9519eee27a None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:47.416510 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb30b5-ea7d-41a5-9416-63d94e7da032 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:47.450657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557924e-7a6f-487e-afcd-ae9fd9c8d9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:47.451315 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557924e-7a6f-487e-afcd-ae9fd9c8d9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.451601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557924e-7a6f-487e-afcd-ae9fd9c8d9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.451882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557924e-7a6f-487e-afcd-ae9fd9c8d9b1 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.452587 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557924e-7a6f-487e-afcd-ae9fd9c8d9b1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:47.452896 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557924e-7a6f-487e-afcd-ae9fd9c8d9b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:47.454325 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557924e-7a6f-487e-afcd-ae9fd9c8d9b1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:47.459306 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 449/895] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jun 7 08:20:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:47.460078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557924e-7a6f-487e-afcd-ae9fd9c8d9b1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:47.462060 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:47.505792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650f976-4c69-4447-a892-6e9519eee27a None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:47.528219 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:47.538510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45352eb-d767-4f6a-af29-e9d144bbb6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:47.539467 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45352eb-d767-4f6a-af29-e9d144bbb6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.539749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45352eb-d767-4f6a-af29-e9d144bbb6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.540025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45352eb-d767-4f6a-af29-e9d144bbb6f3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.540681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45352eb-d767-4f6a-af29-e9d144bbb6f3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:47.542089 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45352eb-d767-4f6a-af29-e9d144bbb6f3 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:47.554829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-a45352eb-d767-4f6a-af29-e9d144bbb6f3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:20:47.554829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:47.554829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:20:47.554829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:20:47.554829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:20:47.554829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:20:47.554829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:20:47.554829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:47.554829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:20:47.554829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:47.554829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:47.554829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:47.554829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:47.554829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:47.554829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:47.554829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:47.554829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:47.555845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:47.555845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:47.555845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:47.555845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:47.555845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:47.555845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:47.555845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:20:47.555845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:20:47.555845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:20:47.555845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:20:47.555845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:47.555845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:47.555845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:47.555845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:47.555845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:47.555845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:47.556677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:47.556677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:47.556677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:47.556677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:47.556677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:47.556677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:47.556677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:47.556677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:47.556677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:47.556677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:47.556677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:20:47.556677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:47.556677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:20:47.556677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:47.556677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 447/896] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jun 7 08:20:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:47.557500 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c65a1-5e6d-414f-bcb5-9f7163f65f37 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.557575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c65a1-5e6d-414f-bcb5-9f7163f65f37 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.557924 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c65a1-5e6d-414f-bcb5-9f7163f65f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.569922 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557924e-7a6f-487e-afcd-ae9fd9c8d9b1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:47.574598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9c3c65a1-5e6d-414f-bcb5-9f7163f65f37 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:47.575825 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9c3c65a1-5e6d-414f-bcb5-9f7163f65f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:47.602268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f00d4f-3696-4b83-b0b6-ca70c282daf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:47.603293 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f00d4f-3696-4b83-b0b6-ca70c282daf4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.603580 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f00d4f-3696-4b83-b0b6-ca70c282daf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.603851 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f00d4f-3696-4b83-b0b6-ca70c282daf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.610936 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f00d4f-3696-4b83-b0b6-ca70c282daf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:47.612377 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f00d4f-3696-4b83-b0b6-ca70c282daf4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:47.627076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-91f00d4f-3696-4b83-b0b6-ca70c282daf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:47.627076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:47.627076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:47.627076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:47.627076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:47.627076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:47.627076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:47.627076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:47.627076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:47.627076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:47.627076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:47.627076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:47.627076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:47.627076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:47.627076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:47.628024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:47.628024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:47.628024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:47.628024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:47.628024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:47.628024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:47.628024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:47.628024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:47.628024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:47.628024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:47.628024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:47.628024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:47.628024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:47.628024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:47.628024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:47.628024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:47.628024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:47.628897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:47.628897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:47.628897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:47.628897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:47.628897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:47.628897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:47.628897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:47.628897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:47.628897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 450/897] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:47.629529 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f00d4f-3696-4b83-b0b6-ca70c282daf4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:47.645456 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 448/898] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:47.646515 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c65a1-5e6d-414f-bcb5-9f7163f65f37 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:47.711920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:47.734184 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f00d4f-3696-4b83-b0b6-ca70c282daf4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:47.756552 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:47.760301 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c65a1-5e6d-414f-bcb5-9f7163f65f37 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:47.767770 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15442636-916f-47c4-b1d3-e211b881f85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:47.768417 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15442636-916f-47c4-b1d3-e211b881f85c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.768695 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15442636-916f-47c4-b1d3-e211b881f85c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.768971 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15442636-916f-47c4-b1d3-e211b881f85c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.769773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15442636-916f-47c4-b1d3-e211b881f85c None admin] RBAC: Adding query filter params (interface=public, service_id=85923d90686a4438b771ad552e7f97ea, region_id=RegionOne) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:47.770082 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15442636-916f-47c4-b1d3-e211b881f85c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:47.771458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15442636-916f-47c4-b1d3-e211b881f85c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:47.778960 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 451/899] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Jun 7 08:20:47 2022] GET /identity/v3/endpoints?service_id=85923d90686a4438b771ad552e7f97ea&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:47.779736 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15442636-916f-47c4-b1d3-e211b881f85c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:47.795201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f135b-d205-4ee8-b387-ca37b12e8972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:47.795863 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f135b-d205-4ee8-b387-ca37b12e8972 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.796146 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f135b-d205-4ee8-b387-ca37b12e8972 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.796424 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f135b-d205-4ee8-b387-ca37b12e8972 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.797174 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f135b-d205-4ee8-b387-ca37b12e8972 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:47.797485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f135b-d205-4ee8-b387-ca37b12e8972 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:47.799123 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f135b-d205-4ee8-b387-ca37b12e8972 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:47.803922 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 449/900] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jun 7 08:20:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:47.804681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f135b-d205-4ee8-b387-ca37b12e8972 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:47.878482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15442636-916f-47c4-b1d3-e211b881f85c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:47.899773 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f135b-d205-4ee8-b387-ca37b12e8972 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:47.911166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd64cb-a00a-4ecc-86e8-afff259bff0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:47.911818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd64cb-a00a-4ecc-86e8-afff259bff0a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.912099 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd64cb-a00a-4ecc-86e8-afff259bff0a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.912376 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd64cb-a00a-4ecc-86e8-afff259bff0a None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.913192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd64cb-a00a-4ecc-86e8-afff259bff0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:47.913502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd64cb-a00a-4ecc-86e8-afff259bff0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:47.914425 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:47.914921 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd64cb-a00a-4ecc-86e8-afff259bff0a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:47.927080 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 452/901] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:47.927870 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd64cb-a00a-4ecc-86e8-afff259bff0a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:47.939715 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb257a9-f7af-4121-90dd-8f01cfff6673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:47.940506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb257a9-f7af-4121-90dd-8f01cfff6673 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:47.940791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb257a9-f7af-4121-90dd-8f01cfff6673 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:47.941101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb257a9-f7af-4121-90dd-8f01cfff6673 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:47.941755 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb257a9-f7af-4121-90dd-8f01cfff6673 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:47.943113 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb257a9-f7af-4121-90dd-8f01cfff6673 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:47.954679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-6eb257a9-f7af-4121-90dd-8f01cfff6673 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 08:20:47.954679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:47.954679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:20:47.954679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:20:47.954679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:20:47.954679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:20:47.954679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:20:47.954679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:47.954679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 08:20:47.954679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:47.954679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:47.954679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:47.954679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:47.954679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:47.954679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:47.954679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:47.954679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:47.955643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:47.955643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:47.955643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:47.955643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:47.955643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:47.955643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:47.955643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:20:47.955643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:20:47.955643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:20:47.955643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:20:47.955643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:47.955643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:47.955643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:47.955643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:47.955643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:47.955643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:47.956459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:47.956459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:47.956459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:47.956459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:47.956459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:47.956459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:47.956459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:47.956459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:47.956459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:47.956459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:47.956459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:20:47.956459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:47.956459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 08:20:47.956459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:47.956459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 450/902] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:20:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:47.957326 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb257a9-f7af-4121-90dd-8f01cfff6673 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:47.980417 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:48.029446 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd64cb-a00a-4ecc-86e8-afff259bff0a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:48.054867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb257a9-f7af-4121-90dd-8f01cfff6673 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:48.062683 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15899b-df6a-4298-9b17-b2e25ca1b194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:48.063346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15899b-df6a-4298-9b17-b2e25ca1b194 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:48.063629 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15899b-df6a-4298-9b17-b2e25ca1b194 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:48.063906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15899b-df6a-4298-9b17-b2e25ca1b194 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:48.064585 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce15899b-df6a-4298-9b17-b2e25ca1b194 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:48.065994 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce15899b-df6a-4298-9b17-b2e25ca1b194 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:48.071456 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 453/903] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Jun 7 08:20:47 2022] GET /identity/v3/services => generated 1268 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:20:48.072228 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15899b-df6a-4298-9b17-b2e25ca1b194 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:48.091572 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95965633-4278-441d-827e-30cd7aa933fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:48.092377 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95965633-4278-441d-827e-30cd7aa933fd None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:48.092660 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95965633-4278-441d-827e-30cd7aa933fd None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:48.093091 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95965633-4278-441d-827e-30cd7aa933fd None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:48.093828 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95965633-4278-441d-827e-30cd7aa933fd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:48.094137 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95965633-4278-441d-827e-30cd7aa933fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:48.095704 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95965633-4278-441d-827e-30cd7aa933fd None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:48.113051 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 451/904] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jun 7 08:20:47 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:48.113051 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95965633-4278-441d-827e-30cd7aa933fd None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:48.165168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:48.237430 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15899b-df6a-4298-9b17-b2e25ca1b194 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:48.248824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95965633-4278-441d-827e-30cd7aa933fd None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:48.253170 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:48.279777 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a76fdb-adea-4d53-b7c9-de67034d7b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:48.280612 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a76fdb-adea-4d53-b7c9-de67034d7b1e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:48.280847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a76fdb-adea-4d53-b7c9-de67034d7b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:48.281394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a76fdb-adea-4d53-b7c9-de67034d7b1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:48.282962 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a76fdb-adea-4d53-b7c9-de67034d7b1e None admin] RBAC: Adding query filter params (role.id=c782c67dcd6a46cc8c5b26a87d6d2dc3, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=bbf0ce396885472bb5d225aa009043f2) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:48.283289 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a76fdb-adea-4d53-b7c9-de67034d7b1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:48.284961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a76fdb-adea-4d53-b7c9-de67034d7b1e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:48.293468 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67cd8f-7065-4af7-855d-1abacf308b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:48.294186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67cd8f-7065-4af7-855d-1abacf308b0c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:48.294401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 454/905] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:48 2022] GET /identity/v3/role_assignments?user.id=bbf0ce396885472bb5d225aa009043f2&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=c782c67dcd6a46cc8c5b26a87d6d2dc3 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:48.294774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67cd8f-7065-4af7-855d-1abacf308b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:48.295057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67cd8f-7065-4af7-855d-1abacf308b0c None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:48.295788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67cd8f-7065-4af7-855d-1abacf308b0c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:48.296100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67cd8f-7065-4af7-855d-1abacf308b0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:48.297616 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67cd8f-7065-4af7-855d-1abacf308b0c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:48.303741 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 452/906] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:20:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:48.309895 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a76fdb-adea-4d53-b7c9-de67034d7b1e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:48.430579 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a76fdb-adea-4d53-b7c9-de67034d7b1e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:48.470582 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6210a-475b-4b95-ae18-0073c9a6d5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:48.471661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6210a-475b-4b95-ae18-0073c9a6d5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:48.472153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6210a-475b-4b95-ae18-0073c9a6d5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:48.472661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6210a-475b-4b95-ae18-0073c9a6d5c9 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:48.473638 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6210a-475b-4b95-ae18-0073c9a6d5c9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:48.474180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6210a-475b-4b95-ae18-0073c9a6d5c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:48.475789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6210a-475b-4b95-ae18-0073c9a6d5c9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:48.482333 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 455/907] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:20:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:48.490545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67cd8f-7065-4af7-855d-1abacf308b0c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:48.594615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67cd8f-7065-4af7-855d-1abacf308b0c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:48.601832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:48.634070 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa837d88-03ed-46f3-8533-d307b4d30fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:48.634781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa837d88-03ed-46f3-8533-d307b4d30fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:48.635063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa837d88-03ed-46f3-8533-d307b4d30fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:48.635339 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa837d88-03ed-46f3-8533-d307b4d30fd0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:48.636132 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa837d88-03ed-46f3-8533-d307b4d30fd0 None admin] RBAC: Adding query filter params (interface=public, service_id=e78201aa6c7f493599eab9ce5fd53f16, region_id=RegionOne) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:48.636442 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa837d88-03ed-46f3-8533-d307b4d30fd0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:48.638452 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa837d88-03ed-46f3-8533-d307b4d30fd0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:48.644158 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 453/908] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Jun 7 08:20:48 2022] GET /identity/v3/endpoints?service_id=e78201aa6c7f493599eab9ce5fd53f16&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:48.649968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6210a-475b-4b95-ae18-0073c9a6d5c9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:48.696230 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:48.756344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6210a-475b-4b95-ae18-0073c9a6d5c9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:48.791670 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97260451-de79-462b-b4bc-abe97175a536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:48.792341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97260451-de79-462b-b4bc-abe97175a536 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:48.792627 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97260451-de79-462b-b4bc-abe97175a536 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:48.792905 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97260451-de79-462b-b4bc-abe97175a536 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:48.793623 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97260451-de79-462b-b4bc-abe97175a536 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:48.795019 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97260451-de79-462b-b4bc-abe97175a536 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:48.801348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 456/909] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Jun 7 08:20:48 2022] GET /identity/v3/services => generated 1268 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:20:50.440059 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f9fd8-2965-4cd2-ba74-0443524dd6e5 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:50.440564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f9fd8-2965-4cd2-ba74-0443524dd6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:50.440564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f9fd8-2965-4cd2-ba74-0443524dd6e5 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:50.442156 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 454/910] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:50.452898 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522726c-07cf-4009-9c6d-0d108beb71dd None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:50.453393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522726c-07cf-4009-9c6d-0d108beb71dd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:50.453471 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522726c-07cf-4009-9c6d-0d108beb71dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:50.464302 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf78709-5bed-4e89-a37c-3d0b7685ec63 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:50.464596 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf78709-5bed-4e89-a37c-3d0b7685ec63 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:50.464872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf78709-5bed-4e89-a37c-3d0b7685ec63 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:50.467240 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 455/911] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:20:50.471728 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7522726c-07cf-4009-9c6d-0d108beb71dd None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:50.472665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392ab2b-cd87-4fbe-a5b3-7cd165003069 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:50.472961 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392ab2b-cd87-4fbe-a5b3-7cd165003069 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:50.473086 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7522726c-07cf-4009-9c6d-0d108beb71dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:50.473270 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392ab2b-cd87-4fbe-a5b3-7cd165003069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:50.490025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-f392ab2b-cd87-4fbe-a5b3-7cd165003069 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:50.491229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f392ab2b-cd87-4fbe-a5b3-7cd165003069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:50.552093 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 457/912] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:20:50.577610 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522726c-07cf-4009-9c6d-0d108beb71dd None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:50.585232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 456/913] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:50.609322 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392ab2b-cd87-4fbe-a5b3-7cd165003069 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:50.614246 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:50.683366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522726c-07cf-4009-9c6d-0d108beb71dd None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:50.718631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f6907-e364-4041-8856-e59a33a530ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:50.719368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f6907-e364-4041-8856-e59a33a530ce None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:50.719652 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f6907-e364-4041-8856-e59a33a530ce None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:50.719926 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f6907-e364-4041-8856-e59a33a530ce None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:50.720578 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f6907-e364-4041-8856-e59a33a530ce None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:50.721977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f6907-e364-4041-8856-e59a33a530ce None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:50.723013 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:50.726867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392ab2b-cd87-4fbe-a5b3-7cd165003069 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:50.734361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-510f6907-e364-4041-8856-e59a33a530ce None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 08:20:50.734361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:50.734361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:20:50.734361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:20:50.734361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:20:50.734361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:20:50.734361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:20:50.734361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:50.734361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 08:20:50.734361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:50.734361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:50.734361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:50.734361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:50.734361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:50.734361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:50.734361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:50.735618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:50.735618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:50.735618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:50.735618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:50.735618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:50.735618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:50.735618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:50.735618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:20:50.735618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:20:50.735618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:20:50.735618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:20:50.735618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:50.735618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:50.735618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:50.735618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:50.735618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:50.736760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:50.736760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:50.736760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:50.736760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:50.736760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:50.736760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:50.736760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:50.736760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:50.736760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:50.736760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:50.736760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:50.736760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:20:50.736760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:50.736760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 08:20:50.736760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:50.736760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 458/914] 10.222.0.17 () {56 vars in 1132 bytes} [Tue Jun 7 08:20:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:50.737767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5baac3-2497-46fc-94d7-09a75f573a3e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:50.737767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5baac3-2497-46fc-94d7-09a75f573a3e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:50.737767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5baac3-2497-46fc-94d7-09a75f573a3e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:50.741304 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 459/915] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:50.742239 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5baac3-2497-46fc-94d7-09a75f573a3e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:50.771427 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0728c25-19e4-4f5e-be6d-462f2e2e7d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:50.772554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0728c25-19e4-4f5e-be6d-462f2e2e7d61 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:50.773084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0728c25-19e4-4f5e-be6d-462f2e2e7d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:50.773612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0728c25-19e4-4f5e-be6d-462f2e2e7d61 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:50.774567 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0728c25-19e4-4f5e-be6d-462f2e2e7d61 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:50.776308 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0728c25-19e4-4f5e-be6d-462f2e2e7d61 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:50.837748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 457/916] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jun 7 08:20:50 2022] POST /identity/v3/services => generated 241 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:20:50.839021 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda2bd6-bcdc-4cff-802e-76186b3b0f26 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:50.839309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda2bd6-bcdc-4cff-802e-76186b3b0f26 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:50.840161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda2bd6-bcdc-4cff-802e-76186b3b0f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:50.866123 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:50.870689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9dda2bd6-bcdc-4cff-802e-76186b3b0f26 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:50.872643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9dda2bd6-bcdc-4cff-802e-76186b3b0f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:50.882007 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5baac3-2497-46fc-94d7-09a75f573a3e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:50.931402 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110449f-1062-4263-b5a2-d454a0803f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:50.932468 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110449f-1062-4263-b5a2-d454a0803f06 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:50.932960 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110449f-1062-4263-b5a2-d454a0803f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:50.933491 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110449f-1062-4263-b5a2-d454a0803f06 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:50.934972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110449f-1062-4263-b5a2-d454a0803f06 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:50.935519 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110449f-1062-4263-b5a2-d454a0803f06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:50.937156 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110449f-1062-4263-b5a2-d454a0803f06 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:50.943027 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 460/917] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jun 7 08:20:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:50.948782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110449f-1062-4263-b5a2-d454a0803f06 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:50.977668 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 458/918] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:50.996169 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda2bd6-bcdc-4cff-802e-76186b3b0f26 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:51.074751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110449f-1062-4263-b5a2-d454a0803f06 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:51.113468 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:51.116990 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79362026-080a-4a1d-918a-ce34a84ab19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:51.124626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79362026-080a-4a1d-918a-ce34a84ab19a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:51.124626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79362026-080a-4a1d-918a-ce34a84ab19a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:51.124626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79362026-080a-4a1d-918a-ce34a84ab19a None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:51.124626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79362026-080a-4a1d-918a-ce34a84ab19a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:51.124626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79362026-080a-4a1d-918a-ce34a84ab19a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:51.124626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79362026-080a-4a1d-918a-ce34a84ab19a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:51.125706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda2bd6-bcdc-4cff-802e-76186b3b0f26 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:51.129153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 461/919] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jun 7 08:20:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:51.129153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743ca67-c5cc-43ac-a87b-e4546022dcf6 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:51.129153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743ca67-c5cc-43ac-a87b-e4546022dcf6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:51.129153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743ca67-c5cc-43ac-a87b-e4546022dcf6 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:51.129684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 462/920] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:51.135873 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54dfcc-7dbf-467b-98f7-20d3bdda4d7f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:51.136243 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54dfcc-7dbf-467b-98f7-20d3bdda4d7f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:51.136532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54dfcc-7dbf-467b-98f7-20d3bdda4d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:51.153055 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1a54dfcc-7dbf-467b-98f7-20d3bdda4d7f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:51.154226 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1a54dfcc-7dbf-467b-98f7-20d3bdda4d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:51.165518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc967085-7f74-4269-b3b4-e2f5537e5388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:51.166344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc967085-7f74-4269-b3b4-e2f5537e5388 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:51.166728 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc967085-7f74-4269-b3b4-e2f5537e5388 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:51.167010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc967085-7f74-4269-b3b4-e2f5537e5388 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:51.171657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:51.183780 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc967085-7f74-4269-b3b4-e2f5537e5388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:51.193370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-bc967085-7f74-4269-b3b4-e2f5537e5388 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:20:51.193370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:51.193370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:51.193370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:51.193370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:51.193370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:51.193370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:51.193370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:51.193370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:51.193370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:51.193370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:51.193370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:51.193370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:51.193370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:51.193370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:51.193370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:51.194361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:51.194361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:51.194361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:51.194361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:51.194361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:51.194361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:51.194361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:51.194361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:51.194361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:51.194361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:51.194361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:51.194361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:51.194361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:51.194361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:51.194361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:51.194361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:51.198282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:51.198282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:51.198282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:51.198282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:51.198282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:51.198282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:51.198282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:51.198282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:51.198282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:51.198282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:51.198282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:20:51.198282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:51.198282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 459/921] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Jun 7 08:20:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:51.198282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc967085-7f74-4269-b3b4-e2f5537e5388 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:51.249664 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 463/922] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:51.250967 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a54dfcc-7dbf-467b-98f7-20d3bdda4d7f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:51.326683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc967085-7f74-4269-b3b4-e2f5537e5388 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:51.367647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6de52f-1181-4f0a-afa2-b2910b64d58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:51.368636 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6de52f-1181-4f0a-afa2-b2910b64d58f None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:51.369147 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6de52f-1181-4f0a-afa2-b2910b64d58f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:51.369622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6de52f-1181-4f0a-afa2-b2910b64d58f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:51.370541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6de52f-1181-4f0a-afa2-b2910b64d58f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:51.372752 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6de52f-1181-4f0a-afa2-b2910b64d58f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:51.374526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a54dfcc-7dbf-467b-98f7-20d3bdda4d7f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:51.410197 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e400d9-0002-47cd-a8d0-b331973dd46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:51.410984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e400d9-0002-47cd-a8d0-b331973dd46c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:51.411389 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e400d9-0002-47cd-a8d0-b331973dd46c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:51.411714 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e400d9-0002-47cd-a8d0-b331973dd46c None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:51.412539 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e400d9-0002-47cd-a8d0-b331973dd46c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:51.412948 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e400d9-0002-47cd-a8d0-b331973dd46c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:51.414460 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e400d9-0002-47cd-a8d0-b331973dd46c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:51.423807 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 464/923] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Jun 7 08:20:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:51.424670 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e400d9-0002-47cd-a8d0-b331973dd46c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:51.442229 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:51.453895 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 460/924] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jun 7 08:20:51 2022] POST /identity/v3/endpoints => generated 347 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:51.455430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6de52f-1181-4f0a-afa2-b2910b64d58f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:51.465378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:51.583132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e400d9-0002-47cd-a8d0-b331973dd46c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:51.602153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6de52f-1181-4f0a-afa2-b2910b64d58f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:51.634075 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01605071-467b-4878-bead-42c69f1a1018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:51.636362 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01605071-467b-4878-bead-42c69f1a1018 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:51.636884 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01605071-467b-4878-bead-42c69f1a1018 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:51.637391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01605071-467b-4878-bead-42c69f1a1018 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:51.638485 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01605071-467b-4878-bead-42c69f1a1018 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:51.640162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01605071-467b-4878-bead-42c69f1a1018 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:51.658273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-01605071-467b-4878-bead-42c69f1a1018 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 08:20:51.658273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:51.658273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:20:51.658273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:20:51.658273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:20:51.658273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:20:51.658273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:20:51.658273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:51.658273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 08:20:51.658273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:51.658273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:51.658273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:51.658273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:51.658273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:51.658273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:51.658273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:51.658273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:51.659316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:51.659316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:51.659316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:51.659316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:51.659316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:51.659316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:51.659316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:20:51.659316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:20:51.659316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:20:51.659316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:20:51.659316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:51.659316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:51.659316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:51.659316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:51.659316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:51.659316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:51.660118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:51.660118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:51.660118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:51.660118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:51.660118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:51.660118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:51.660118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:51.660118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:51.660118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:51.660118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:51.660118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:20:51.660118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:51.660118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 08:20:51.660118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:51.667757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 465/925] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:20:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:51.669702 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01605071-467b-4878-bead-42c69f1a1018 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:51.697028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c080fe2-6112-4993-a866-422def98e7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:51.698502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c080fe2-6112-4993-a866-422def98e7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:51.698502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c080fe2-6112-4993-a866-422def98e7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:51.698502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c080fe2-6112-4993-a866-422def98e7b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:51.717820 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c080fe2-6112-4993-a866-422def98e7b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:51.719361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c080fe2-6112-4993-a866-422def98e7b4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:51.747056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-3c080fe2-6112-4993-a866-422def98e7b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:51.747056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:51.747056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:51.747056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:51.747056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:51.747056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:51.747056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:51.747056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:51.747056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:51.747056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:51.747056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:51.747056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:51.747056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:51.747056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:51.747056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:51.748018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:51.748018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:51.748018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:51.748018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:51.748018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:51.748018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:51.748018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:51.748018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:51.748018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:51.748018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:51.748018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:51.748018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:51.748018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:51.748018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:51.748018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:51.748018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:51.748018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:51.748927 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:51.748927 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:51.748927 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:51.748927 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:51.748927 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 461/926] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:51.756246 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c080fe2-6112-4993-a866-422def98e7b4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:51.803364 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:51.829564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01605071-467b-4878-bead-42c69f1a1018 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:51.833458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:51.872632 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8738c-b275-4c01-b7ec-7db6dd1bf74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:51.873768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8738c-b275-4c01-b7ec-7db6dd1bf74d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:51.874189 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8738c-b275-4c01-b7ec-7db6dd1bf74d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:51.874469 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8738c-b275-4c01-b7ec-7db6dd1bf74d None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:51.875186 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8738c-b275-4c01-b7ec-7db6dd1bf74d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:51.875495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8738c-b275-4c01-b7ec-7db6dd1bf74d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:51.876051 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c080fe2-6112-4993-a866-422def98e7b4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:51.876856 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8738c-b275-4c01-b7ec-7db6dd1bf74d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:51.883043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 466/927] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:20:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:51.888979 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8738c-b275-4c01-b7ec-7db6dd1bf74d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:51.924940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97484dec-915a-4218-9949-41d691e7b7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:51.925962 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97484dec-915a-4218-9949-41d691e7b7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:51.926450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97484dec-915a-4218-9949-41d691e7b7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:51.926921 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97484dec-915a-4218-9949-41d691e7b7e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:51.927948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97484dec-915a-4218-9949-41d691e7b7e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:51.928461 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97484dec-915a-4218-9949-41d691e7b7e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:51.930269 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97484dec-915a-4218-9949-41d691e7b7e2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:51.943730 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 462/928] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:51.949051 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97484dec-915a-4218-9949-41d691e7b7e2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:51.997924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8738c-b275-4c01-b7ec-7db6dd1bf74d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:52.033517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e5ca2-6b0a-4799-b666-d8481af31c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:52.034212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e5ca2-6b0a-4799-b666-d8481af31c3c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:52.034497 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e5ca2-6b0a-4799-b666-d8481af31c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:52.034777 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e5ca2-6b0a-4799-b666-d8481af31c3c None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:52.035505 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e5ca2-6b0a-4799-b666-d8481af31c3c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:52.035817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e5ca2-6b0a-4799-b666-d8481af31c3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:52.037313 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e5ca2-6b0a-4799-b666-d8481af31c3c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:52.047611 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 467/929] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Jun 7 08:20:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:52.048945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e5ca2-6b0a-4799-b666-d8481af31c3c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:52.058573 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:52.067680 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97484dec-915a-4218-9949-41d691e7b7e2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:52.083260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:52.105574 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2423bd1-447e-4510-83ed-343bc4101149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:52.106708 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2423bd1-447e-4510-83ed-343bc4101149 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:52.107181 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2423bd1-447e-4510-83ed-343bc4101149 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:52.107732 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2423bd1-447e-4510-83ed-343bc4101149 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:52.115846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2423bd1-447e-4510-83ed-343bc4101149 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:52.117752 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2423bd1-447e-4510-83ed-343bc4101149 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:52.134230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-f2423bd1-447e-4510-83ed-343bc4101149 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:20:52.134230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:52.134230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:52.134230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:52.134230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:52.134230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:52.134230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:52.134230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:52.134230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:52.134230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:52.134230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:52.134230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:52.134230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:52.134230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:52.134230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:52.134230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:52.135226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:52.135226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:52.135226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:52.135226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:52.135226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:52.135226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:52.135226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:52.135226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:52.135226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:52.135226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:52.135226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:52.135226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:52.135226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:52.135226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:52.135226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:52.135226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:52.135226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:52.135226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:52.136159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:52.136159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:52.136159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:52.136159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:52.136159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:52.136159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:52.136159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:52.136159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:52.136159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:20:52.136159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:52.139485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 463/930] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jun 7 08:20:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:52.145315 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2423bd1-447e-4510-83ed-343bc4101149 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:52.161092 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e5ca2-6b0a-4799-b666-d8481af31c3c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:52.208503 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b71f88-dfe2-4536-983b-80eb80defb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:52.209251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b71f88-dfe2-4536-983b-80eb80defb91 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:52.209539 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b71f88-dfe2-4536-983b-80eb80defb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:52.209819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b71f88-dfe2-4536-983b-80eb80defb91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:52.210564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b71f88-dfe2-4536-983b-80eb80defb91 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:52.211999 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b71f88-dfe2-4536-983b-80eb80defb91 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:52.265365 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2423bd1-447e-4510-83ed-343bc4101149 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:52.300342 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fbb0c-f07b-4330-bfd9-591eafec6ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:52.301420 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fbb0c-f07b-4330-bfd9-591eafec6ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:52.302115 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fbb0c-f07b-4330-bfd9-591eafec6ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:52.302573 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fbb0c-f07b-4330-bfd9-591eafec6ac8 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:52.303570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9fbb0c-f07b-4330-bfd9-591eafec6ac8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:52.304039 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9fbb0c-f07b-4330-bfd9-591eafec6ac8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:52.305752 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9fbb0c-f07b-4330-bfd9-591eafec6ac8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:52.324121 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 464/931] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Jun 7 08:20:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:52.330453 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fbb0c-f07b-4330-bfd9-591eafec6ac8 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:52.361481 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:52.393338 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 468/932] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jun 7 08:20:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:52.445406 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fbb0c-f07b-4330-bfd9-591eafec6ac8 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:52.490331 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e291b6-39a7-4d77-a6c9-f3d846afd015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:52.491937 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e291b6-39a7-4d77-a6c9-f3d846afd015 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:52.492503 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e291b6-39a7-4d77-a6c9-f3d846afd015 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:52.492978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e291b6-39a7-4d77-a6c9-f3d846afd015 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:52.505553 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e291b6-39a7-4d77-a6c9-f3d846afd015 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:52.507403 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e291b6-39a7-4d77-a6c9-f3d846afd015 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:52.526791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-98e291b6-39a7-4d77-a6c9-f3d846afd015 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:52.526791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:52.526791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:52.526791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:52.526791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:52.526791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:52.526791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:52.526791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:52.526791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:52.526791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:52.526791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:52.526791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:52.526791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:52.526791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:52.526791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:52.527749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:52.527749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:52.527749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:52.527749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:52.527749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:52.527749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:52.527749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:52.527749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:52.527749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:52.527749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:52.527749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:52.527749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:52.527749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:52.527749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:52.527749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:52.527749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:52.527749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:52.529306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:52.529306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:52.529306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:52.529306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:52.531678 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 465/933] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:52.537698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b71f88-dfe2-4536-983b-80eb80defb91 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:52.604562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:52.647554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b71f88-dfe2-4536-983b-80eb80defb91 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:52.683013 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e826c-828f-4620-be30-fe78803d7250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:52.684016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e826c-828f-4620-be30-fe78803d7250 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:52.684682 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e826c-828f-4620-be30-fe78803d7250 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:52.685175 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e826c-828f-4620-be30-fe78803d7250 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:52.686121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e826c-828f-4620-be30-fe78803d7250 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:52.686613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e826c-828f-4620-be30-fe78803d7250 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:52.688223 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e826c-828f-4620-be30-fe78803d7250 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:52.703236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 469/934] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:52.708446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e291b6-39a7-4d77-a6c9-f3d846afd015 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:52.793080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:52.812004 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e291b6-39a7-4d77-a6c9-f3d846afd015 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:52.850721 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a7eac-c947-4d35-bad6-35da43c9de81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:52.851955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a7eac-c947-4d35-bad6-35da43c9de81 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:52.852232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a7eac-c947-4d35-bad6-35da43c9de81 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:52.852504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a7eac-c947-4d35-bad6-35da43c9de81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:52.860555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a7eac-c947-4d35-bad6-35da43c9de81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:52.862120 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a7eac-c947-4d35-bad6-35da43c9de81 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:52.877100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-235a7eac-c947-4d35-bad6-35da43c9de81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:52.877100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:52.877100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:52.877100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:52.877100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:52.877100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:52.877100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:52.877100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:52.877100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:52.877100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:52.877100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:52.877100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:52.877100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:52.877100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:52.877100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:52.878072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:52.878072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:52.878072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:52.878072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:52.878072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:52.878072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:52.878072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:52.878072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:52.878072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:52.878072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:52.878072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:52.878072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:52.878072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:52.878072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:52.878072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:52.878072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:52.878072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:52.878966 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:52.878966 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:52.878966 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:52.878966 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:52.878966 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:52.878966 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:52.878966 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:52.878966 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:52.878966 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 466/935] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:52.884405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e826c-828f-4620-be30-fe78803d7250 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:52.992988 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e826c-828f-4620-be30-fe78803d7250 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:53.018196 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:53.030138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92b8d7-e92c-45f5-912d-33ab33acb5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:53.030991 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92b8d7-e92c-45f5-912d-33ab33acb5ce None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:53.031416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92b8d7-e92c-45f5-912d-33ab33acb5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:53.035972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92b8d7-e92c-45f5-912d-33ab33acb5ce None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:53.035972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92b8d7-e92c-45f5-912d-33ab33acb5ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:53.035972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92b8d7-e92c-45f5-912d-33ab33acb5ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:53.035972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92b8d7-e92c-45f5-912d-33ab33acb5ce None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:53.046856 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 470/936] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:53.054287 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a7eac-c947-4d35-bad6-35da43c9de81 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:53.154672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a7eac-c947-4d35-bad6-35da43c9de81 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:53.182937 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c369d-e368-49bf-b7fe-3cd0e6c04952 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:53.183223 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c369d-e368-49bf-b7fe-3cd0e6c04952 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:53.183507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c369d-e368-49bf-b7fe-3cd0e6c04952 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:53.189101 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 471/937] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:20:53.190929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59118e-307d-4ec8-9a0e-a6b14f4c87db None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:53.191223 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59118e-307d-4ec8-9a0e-a6b14f4c87db None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:53.191508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59118e-307d-4ec8-9a0e-a6b14f4c87db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:53.203992 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4a8ae-56dd-4bd4-bdb4-78dd688fc545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:53.205183 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4a8ae-56dd-4bd4-bdb4-78dd688fc545 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:53.205672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4a8ae-56dd-4bd4-bdb4-78dd688fc545 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:53.206130 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4a8ae-56dd-4bd4-bdb4-78dd688fc545 None admin] PATH_INFO: `/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/bbf0ce396885472bb5d225aa009043f2/roles/c782c67dcd6a46cc8c5b26a87d6d2dc3` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:53.208583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4a8ae-56dd-4bd4-bdb4-78dd688fc545 None admin] RBAC: Authorizing `identity:create_grant(project_id=afc8781e5f864d4a9374300ee9b5193b, user_id=bbf0ce396885472bb5d225aa009043f2, role_id=c782c67dcd6a46cc8c5b26a87d6d2dc3)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:53.208706 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0c59118e-307d-4ec8-9a0e-a6b14f4c87db None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:53.209868 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0c59118e-307d-4ec8-9a0e-a6b14f4c87db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:53.210588 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4a8ae-56dd-4bd4-bdb4-78dd688fc545 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:53.272835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:53.279856 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 467/938] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:20:53 2022] PUT /identity/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/bbf0ce396885472bb5d225aa009043f2/roles/c782c67dcd6a46cc8c5b26a87d6d2dc3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:20:53.289305 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 472/939] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:20:53.306334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4a8ae-56dd-4bd4-bdb4-78dd688fc545 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:53.426666 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4a8ae-56dd-4bd4-bdb4-78dd688fc545 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:53.464822 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcaf058-5014-4deb-88c6-c3379e5006b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:53.466131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaf058-5014-4deb-88c6-c3379e5006b6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:53.466699 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaf058-5014-4deb-88c6-c3379e5006b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:53.467179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaf058-5014-4deb-88c6-c3379e5006b6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:53.468070 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcaf058-5014-4deb-88c6-c3379e5006b6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:53.469694 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcaf058-5014-4deb-88c6-c3379e5006b6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:53.483314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-efcaf058-5014-4deb-88c6-c3379e5006b6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:20:53.483314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:53.483314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:20:53.483314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:20:53.483314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:20:53.483314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:20:53.483314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:20:53.483314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:53.483314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:20:53.483314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:53.483314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:53.483314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:53.483314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:53.483314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:53.483314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:53.483314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:53.483314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:53.484333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:53.484333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:53.484333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:53.484333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:53.484333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:53.484333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:53.484333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:20:53.484333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:20:53.484333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:20:53.484333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:20:53.484333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:53.484333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:53.484333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:53.484333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:53.484333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:53.484333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:53.485209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:53.485209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:53.485209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:53.485209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:53.485209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:53.485209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:53.485209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:53.485209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:53.485209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:53.485209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:53.485209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:20:53.485209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:53.485209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:20:53.485209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:53.488095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 468/940] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jun 7 08:20:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:53.495505 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59118e-307d-4ec8-9a0e-a6b14f4c87db None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:53.605104 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59118e-307d-4ec8-9a0e-a6b14f4c87db None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:53.622635 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:53.645221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f900d-659b-4255-91e5-37e7b8255509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:53.646243 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f900d-659b-4255-91e5-37e7b8255509 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:53.646729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f900d-659b-4255-91e5-37e7b8255509 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:53.647221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f900d-659b-4255-91e5-37e7b8255509 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:53.648134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f900d-659b-4255-91e5-37e7b8255509 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:53.648649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f900d-659b-4255-91e5-37e7b8255509 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:53.650259 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f900d-659b-4255-91e5-37e7b8255509 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:53.656482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 473/941] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jun 7 08:20:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:53.662125 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcaf058-5014-4deb-88c6-c3379e5006b6 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:53.732688 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:53.772057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcaf058-5014-4deb-88c6-c3379e5006b6 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:53.807573 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efd0e1-f8f5-4c84-9e11-c3de055f04c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:53.808234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efd0e1-f8f5-4c84-9e11-c3de055f04c9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:53.808515 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efd0e1-f8f5-4c84-9e11-c3de055f04c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:53.808787 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efd0e1-f8f5-4c84-9e11-c3de055f04c9 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:53.810063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efd0e1-f8f5-4c84-9e11-c3de055f04c9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:53.810450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efd0e1-f8f5-4c84-9e11-c3de055f04c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:53.811841 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efd0e1-f8f5-4c84-9e11-c3de055f04c9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:53.817096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 469/942] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jun 7 08:20:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:53.823031 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f900d-659b-4255-91e5-37e7b8255509 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:53.898742 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019f22b-90aa-425e-b1d7-c50a755b13aa None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:53.899038 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019f22b-90aa-425e-b1d7-c50a755b13aa None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:53.899214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019f22b-90aa-425e-b1d7-c50a755b13aa None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:53.900847 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 470/943] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:20:53.906134 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85039d-1a2c-44a2-8ce5-398d7ec8ad8e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:53.906538 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85039d-1a2c-44a2-8ce5-398d7ec8ad8e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:53.906941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85039d-1a2c-44a2-8ce5-398d7ec8ad8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:53.927850 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-cb85039d-1a2c-44a2-8ce5-398d7ec8ad8e None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:53.929496 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cb85039d-1a2c-44a2-8ce5-398d7ec8ad8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:53.933514 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f900d-659b-4255-91e5-37e7b8255509 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:53.969739 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5362b-8f86-49f1-8e4c-a13f56638713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:53.970850 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5362b-8f86-49f1-8e4c-a13f56638713 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:53.971444 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5362b-8f86-49f1-8e4c-a13f56638713 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:53.971948 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5362b-8f86-49f1-8e4c-a13f56638713 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:53.973023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5362b-8f86-49f1-8e4c-a13f56638713 None admin] RBAC: Adding query filter params (interface=public, service_id=be4491ad274e4214be45b5f282d990b8, region_id=RegionOne) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:53.973557 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5362b-8f86-49f1-8e4c-a13f56638713 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:53.975124 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5362b-8f86-49f1-8e4c-a13f56638713 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:53.982173 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 474/944] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Jun 7 08:20:53 2022] GET /identity/v3/endpoints?service_id=be4491ad274e4214be45b5f282d990b8&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:53.986867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5362b-8f86-49f1-8e4c-a13f56638713 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:54.021043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 471/945] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:20:54.036665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85039d-1a2c-44a2-8ce5-398d7ec8ad8e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:54.039656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:54.117613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:54.121043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5362b-8f86-49f1-8e4c-a13f56638713 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:54.166209 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bdb1c9-14f3-4414-9ec9-7da0cd7d353d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:54.167910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdb1c9-14f3-4414-9ec9-7da0cd7d353d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:54.168416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdb1c9-14f3-4414-9ec9-7da0cd7d353d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:54.169446 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdb1c9-14f3-4414-9ec9-7da0cd7d353d None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:54.170090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85039d-1a2c-44a2-8ce5-398d7ec8ad8e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:54.170620 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bdb1c9-14f3-4414-9ec9-7da0cd7d353d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:54.175134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bdb1c9-14f3-4414-9ec9-7da0cd7d353d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:54.197022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 475/946] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Jun 7 08:20:53 2022] GET /identity/v3/services => generated 1497 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:20:54.225614 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e7680-00fe-485f-b6f2-c441da7e9cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:54.226433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e7680-00fe-485f-b6f2-c441da7e9cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:54.226713 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e7680-00fe-485f-b6f2-c441da7e9cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:54.226986 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e7680-00fe-485f-b6f2-c441da7e9cc3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:54.227650 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e7680-00fe-485f-b6f2-c441da7e9cc3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:54.229170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e7680-00fe-485f-b6f2-c441da7e9cc3 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:54.243613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-f81e7680-00fe-485f-b6f2-c441da7e9cc3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 08:20:54.243613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:54.243613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:20:54.243613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:20:54.243613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:20:54.243613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:20:54.243613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:20:54.243613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:54.243613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 08:20:54.243613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:54.243613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:54.243613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:54.243613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:54.243613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:54.243613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:54.243613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:54.243613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:54.244686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:54.244686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:54.244686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:54.244686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:54.244686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:54.244686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:54.244686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:20:54.244686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:20:54.244686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:20:54.244686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:20:54.244686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:54.244686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:54.244686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:54.244686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:54.244686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:54.244686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:54.246096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:54.246096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:54.246096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:54.246096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:54.246096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:54.246096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:54.246096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:54.246096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:54.246096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:54.246096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:54.246096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:20:54.246096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:54.246096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 08:20:54.246096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:54.246096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 472/947] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:54.253782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bdb1c9-14f3-4414-9ec9-7da0cd7d353d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:54.374628 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bdb1c9-14f3-4414-9ec9-7da0cd7d353d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:54.377278 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:54.415448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059f600-e0f5-44c1-872c-7ae35ef57eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:54.416145 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059f600-e0f5-44c1-872c-7ae35ef57eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:54.416482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059f600-e0f5-44c1-872c-7ae35ef57eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:54.416702 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059f600-e0f5-44c1-872c-7ae35ef57eb8 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:54.417463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059f600-e0f5-44c1-872c-7ae35ef57eb8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:54.417776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059f600-e0f5-44c1-872c-7ae35ef57eb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:54.419775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059f600-e0f5-44c1-872c-7ae35ef57eb8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:54.427608 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 476/948] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jun 7 08:20:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:54.433575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e7680-00fe-485f-b6f2-c441da7e9cc3 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:54.544293 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e7680-00fe-485f-b6f2-c441da7e9cc3 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:54.567934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:54.580323 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247fc4c-1667-4367-bbb9-83c6c267a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:54.581084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247fc4c-1667-4367-bbb9-83c6c267a77b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:54.581387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247fc4c-1667-4367-bbb9-83c6c267a77b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:54.581642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247fc4c-1667-4367-bbb9-83c6c267a77b None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:54.582350 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8247fc4c-1667-4367-bbb9-83c6c267a77b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:54.582655 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8247fc4c-1667-4367-bbb9-83c6c267a77b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:54.584029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8247fc4c-1667-4367-bbb9-83c6c267a77b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:54.588959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 473/949] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jun 7 08:20:54 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:54.759310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05508c9c-9c5c-46ac-9af9-87f0e16d99dc None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:54.759811 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05508c9c-9c5c-46ac-9af9-87f0e16d99dc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:54.759811 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05508c9c-9c5c-46ac-9af9-87f0e16d99dc None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:54.761413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 477/950] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:54.767354 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d68c18-8431-44ba-b26e-90d4ba49912c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:54.767874 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d68c18-8431-44ba-b26e-90d4ba49912c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:54.767952 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d68c18-8431-44ba-b26e-90d4ba49912c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:54.788238 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-94d68c18-8431-44ba-b26e-90d4ba49912c None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:54.789859 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-94d68c18-8431-44ba-b26e-90d4ba49912c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:54.861765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 474/951] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:54.885266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05508c9c-9c5c-46ac-9af9-87f0e16d99dc None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:54.992172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05508c9c-9c5c-46ac-9af9-87f0e16d99dc None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:55.028004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc90bc-00d3-4b09-b3db-491a2ffcb84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:55.028682 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc90bc-00d3-4b09-b3db-491a2ffcb84f None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:55.028971 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc90bc-00d3-4b09-b3db-491a2ffcb84f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:55.029277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc90bc-00d3-4b09-b3db-491a2ffcb84f None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:55.030041 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc90bc-00d3-4b09-b3db-491a2ffcb84f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:55.031496 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc90bc-00d3-4b09-b3db-491a2ffcb84f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:55.032921 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: WARNING keystone.server.flask.common [None req-66cc90bc-00d3-4b09-b3db-491a2ffcb84f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 07 08:20:55.087191 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.sql.core [None req-66cc90bc-00d3-4b09-b3db-491a2ffcb84f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 08:20:55.087191 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 07 08:20:55.087191 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [parameters: {'id': 'ed57dbb93dc24cac867f827f5c27943c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 07 08:20:55.087191 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80358) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 08:20:55.095192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-66cc90bc-00d3-4b09-b3db-491a2ffcb84f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ed57dbb93dc24cac867f827f5c27943c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ed57dbb93dc24cac867f827f5c27943c. Jun 07 08:20:55.095192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:55.095192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 08:20:55.095192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 08:20:55.095192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 08:20:55.095192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 08:20:55.095192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 08:20:55.095192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 08:20:55.095192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 08:20:55.095192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application conn.query(q) Jun 07 08:20:55.095192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 08:20:55.095192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 08:20:55.095192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 08:20:55.095964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result.read() Jun 07 08:20:55.095964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 08:20:55.095964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 08:20:55.095964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 08:20:55.095964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 08:20:55.095964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 08:20:55.095964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 08:20:55.095964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 08:20:55.095964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 08:20:55.095964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 08:20:55.095964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:55.095964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 08:20:55.095964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:55.095964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:55.095964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 08:20:55.095964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 08:20:55.095964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 07 08:20:55.095964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 07 08:20:55.096907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 08:20:55.096907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application next(self.gen) Jun 07 08:20:55.096907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 08:20:55.096907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application yield resource Jun 07 08:20:55.096907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 08:20:55.096907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application next(self.gen) Jun 07 08:20:55.096907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 08:20:55.096907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 08:20:55.096907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 08:20:55.096907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 08:20:55.096907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 08:20:55.096907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise self.value Jun 07 08:20:55.096907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 08:20:55.096907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 08:20:55.096907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 08:20:55.096907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application session.commit() Jun 07 08:20:55.096907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 08:20:55.097820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 08:20:55.097820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 08:20:55.097820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 08:20:55.097820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 08:20:55.097820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.session.flush() Jun 07 08:20:55.097820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 08:20:55.097820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 08:20:55.097820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 08:20:55.097820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 08:20:55.097820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 08:20:55.097820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application compat.raise_( Jun 07 08:20:55.097820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 08:20:55.097820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception Jun 07 08:20:55.097820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 08:20:55.097820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 08:20:55.097820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 08:20:55.097820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 08:20:55.098711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 08:20:55.098711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 08:20:55.098711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 08:20:55.098711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 08:20:55.098711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 07 08:20:55.098711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 07 08:20:55.098711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 08:20:55.098711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 08:20:55.098711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 08:20:55.098711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 08:20:55.098711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 08:20:55.098711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 08:20:55.098711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 08:20:55.098711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 08:20:55.098711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 08:20:55.099502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 08:20:55.099502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 08:20:55.099502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception Jun 07 08:20:55.099502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 08:20:55.099502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 08:20:55.099502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 08:20:55.099502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 08:20:55.099502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 08:20:55.099502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 08:20:55.099502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 08:20:55.099502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application conn.query(q) Jun 07 08:20:55.099502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 08:20:55.099502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 08:20:55.099502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 08:20:55.099502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result.read() Jun 07 08:20:55.099502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 08:20:55.099502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 08:20:55.100346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 08:20:55.100346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 08:20:55.100346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 08:20:55.100346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 08:20:55.100346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 08:20:55.100346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 08:20:55.100346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 08:20:55.100346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 07 08:20:55.100346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [parameters: {'id': 'ed57dbb93dc24cac867f827f5c27943c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 07 08:20:55.100346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 08:20:55.100346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:55.100346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:55.100346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:55.100346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:55.100346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 07 08:20:55.107236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 07 08:20:55.107236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 08:20:55.107236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 08:20:55.107236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 07 08:20:55.107236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:55.107236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:55.107236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:55.107236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:55.107236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:55.107236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:55.107236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:55.107236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:55.107236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:55.107236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:55.107236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:55.107236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:55.107236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:55.108177 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:55.108177 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 07 08:20:55.108177 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 07 08:20:55.108177 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:55.108177 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:55.108177 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 07 08:20:55.108177 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 07 08:20:55.108177 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ed57dbb93dc24cac867f827f5c27943c. Jun 07 08:20:55.108177 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:55.108177 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 478/952] 10.222.0.17 () {60 vars in 1140 bytes} [Tue Jun 7 08:20:54 2022] POST /identity/v3/projects => generated 262 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:55.108736 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d68c18-8431-44ba-b26e-90d4ba49912c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:55.211388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d68c18-8431-44ba-b26e-90d4ba49912c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:55.228939 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:55.247315 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e0cb1-bbcb-4628-a769-a14289f17942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:55.248109 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e0cb1-bbcb-4628-a769-a14289f17942 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:55.248390 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e0cb1-bbcb-4628-a769-a14289f17942 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:55.248666 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e0cb1-bbcb-4628-a769-a14289f17942 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:55.255605 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e0cb1-bbcb-4628-a769-a14289f17942 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:55.257071 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e0cb1-bbcb-4628-a769-a14289f17942 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:55.271483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-a33e0cb1-bbcb-4628-a769-a14289f17942 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:55.271483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:55.271483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:55.271483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:55.271483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:55.271483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:55.271483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:55.271483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:55.271483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:55.271483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:55.271483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:55.271483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:55.271483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:55.271483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:55.271483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:55.272446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:55.272446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:55.272446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:55.272446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:55.272446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:55.272446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:55.272446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:55.272446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:55.272446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:55.272446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:55.272446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:55.272446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:55.272446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:55.272446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:55.272446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:55.272446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:55.272446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:55.273430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:55.273430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:55.273430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:55.273430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:55.273430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:55.273430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:55.273430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:55.273430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:55.275219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 475/953] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:55.281909 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc90bc-00d3-4b09-b3db-491a2ffcb84f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:55.311676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:55.394530 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc90bc-00d3-4b09-b3db-491a2ffcb84f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:55.430091 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe73fe-8bff-417f-965f-c206a709352f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:55.431133 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe73fe-8bff-417f-965f-c206a709352f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:55.431633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe73fe-8bff-417f-965f-c206a709352f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:55.432119 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe73fe-8bff-417f-965f-c206a709352f None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:55.433129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe73fe-8bff-417f-965f-c206a709352f None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:55.433676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe73fe-8bff-417f-965f-c206a709352f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:55.435539 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe73fe-8bff-417f-965f-c206a709352f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:55.456267 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 479/954] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jun 7 08:20:55 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:55.682386 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232e67f-f2f1-4d3d-af4e-9571c4c59863 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:55.682732 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232e67f-f2f1-4d3d-af4e-9571c4c59863 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:55.682986 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232e67f-f2f1-4d3d-af4e-9571c4c59863 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:55.684659 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 476/955] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:55.690173 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b803a-caf0-4401-9165-e577892b2d52 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:55.690709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b803a-caf0-4401-9165-e577892b2d52 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:55.691182 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b803a-caf0-4401-9165-e577892b2d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:55.709797 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4e5b803a-caf0-4401-9165-e577892b2d52 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:55.711598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4e5b803a-caf0-4401-9165-e577892b2d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:55.782920 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 480/956] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:55.807033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232e67f-f2f1-4d3d-af4e-9571c4c59863 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:55.923776 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232e67f-f2f1-4d3d-af4e-9571c4c59863 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:55.959690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfc615-01e0-445a-8ca9-63bf82e10dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:55.960427 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfc615-01e0-445a-8ca9-63bf82e10dfc None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:55.960709 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfc615-01e0-445a-8ca9-63bf82e10dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:55.960982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfc615-01e0-445a-8ca9-63bf82e10dfc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:55.972623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdfc615-01e0-445a-8ca9-63bf82e10dfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:20:55.980788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-bbdfc615-01e0-445a-8ca9-63bf82e10dfc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:20:55.980788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:55.980788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:55.980788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:55.980788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:55.980788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:55.980788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:55.980788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:55.980788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:55.980788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:55.980788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:55.980788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:55.980788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:20:55.980788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:20:55.980788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:20:55.980788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:20:55.981769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:20:55.981769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:20:55.981769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:20:55.981769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:20:55.981769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:55.981769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:55.981769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:55.981769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:55.981769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:55.981769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:55.981769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:55.981769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:55.981769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:55.981769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:55.981769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:55.981769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:55.982601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:55.982601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:55.982601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:55.982601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:55.982601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:20:55.982601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:20:55.982601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:20:55.982601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:20:55.982601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:20:55.982601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:20:55.982601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:20:55.982601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:55.982601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 477/957] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Jun 7 08:20:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:55.988598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b803a-caf0-4401-9165-e577892b2d52 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:56.005159 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:56.094435 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b803a-caf0-4401-9165-e577892b2d52 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:56.130072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222bfd4-f8c2-4116-9d21-5584d1d53625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:56.131112 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222bfd4-f8c2-4116-9d21-5584d1d53625 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:56.131620 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222bfd4-f8c2-4116-9d21-5584d1d53625 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:56.132115 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222bfd4-f8c2-4116-9d21-5584d1d53625 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:56.133094 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8222bfd4-f8c2-4116-9d21-5584d1d53625 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:56.133610 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8222bfd4-f8c2-4116-9d21-5584d1d53625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:56.135182 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8222bfd4-f8c2-4116-9d21-5584d1d53625 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:56.149189 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 481/958] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Jun 7 08:20:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:56.151442 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfc615-01e0-445a-8ca9-63bf82e10dfc None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:56.209387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:56.277643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfc615-01e0-445a-8ca9-63bf82e10dfc None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:56.313266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a8e53-3ca6-4e33-b910-f28f15f23b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:56.314060 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a8e53-3ca6-4e33-b910-f28f15f23b42 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:56.314344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a8e53-3ca6-4e33-b910-f28f15f23b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:56.314623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a8e53-3ca6-4e33-b910-f28f15f23b42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:56.325686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a8e53-3ca6-4e33-b910-f28f15f23b42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:56.327213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a8e53-3ca6-4e33-b910-f28f15f23b42 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:56.344990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-1d7a8e53-3ca6-4e33-b910-f28f15f23b42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:56.344990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:56.344990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:56.344990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:56.344990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:56.344990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:56.344990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:56.344990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:56.344990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:56.344990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:56.344990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:56.344990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:56.344990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:56.344990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:56.344990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:56.345925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:56.345925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:56.345925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:56.345925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:56.345925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:56.345925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:56.345925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:56.345925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:56.345925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:56.345925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:56.345925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:56.345925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:56.345925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:56.345925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:56.345925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:56.345925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:56.345925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:56.346813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:56.346813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:56.346813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:56.346813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:56.350893 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 478/959] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:56.352235 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222bfd4-f8c2-4116-9d21-5584d1d53625 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:56.437978 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:56.465741 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222bfd4-f8c2-4116-9d21-5584d1d53625 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:56.495001 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781b9b6-92a4-4c26-83ad-d9a302e8cdc8 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:56.495338 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781b9b6-92a4-4c26-83ad-d9a302e8cdc8 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:56.495740 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781b9b6-92a4-4c26-83ad-d9a302e8cdc8 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:56.498634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 479/960] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:20:56.502926 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072ce0c-89a5-45f4-b9f6-62e4f22d3011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:56.503437 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cce77-fc23-4174-9816-5f157bb65b5e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:56.503616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072ce0c-89a5-45f4-b9f6-62e4f22d3011 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:56.503713 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cce77-fc23-4174-9816-5f157bb65b5e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:56.503863 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072ce0c-89a5-45f4-b9f6-62e4f22d3011 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:56.503965 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cce77-fc23-4174-9816-5f157bb65b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:56.504143 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072ce0c-89a5-45f4-b9f6-62e4f22d3011 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:56.504866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c072ce0c-89a5-45f4-b9f6-62e4f22d3011 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:56.505205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c072ce0c-89a5-45f4-b9f6-62e4f22d3011 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:56.506584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c072ce0c-89a5-45f4-b9f6-62e4f22d3011 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:56.520789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d60cce77-fc23-4174-9816-5f157bb65b5e None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:56.522002 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d60cce77-fc23-4174-9816-5f157bb65b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:56.524876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 482/961] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:56.526112 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072ce0c-89a5-45f4-b9f6-62e4f22d3011 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:56.603941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 480/962] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:20:56.622895 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cce77-fc23-4174-9816-5f157bb65b5e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:56.644126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:56.659386 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072ce0c-89a5-45f4-b9f6-62e4f22d3011 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:56.709307 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09baea47-d54e-4b19-92a2-9f49f91d05c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:56.710604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baea47-d54e-4b19-92a2-9f49f91d05c8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:56.711294 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baea47-d54e-4b19-92a2-9f49f91d05c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:56.711716 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baea47-d54e-4b19-92a2-9f49f91d05c8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:56.716071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:56.723094 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09baea47-d54e-4b19-92a2-9f49f91d05c8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:56.725020 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09baea47-d54e-4b19-92a2-9f49f91d05c8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:56.744467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-09baea47-d54e-4b19-92a2-9f49f91d05c8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:20:56.744467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:56.744467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:56.744467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:56.744467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:56.744467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:56.744467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:56.744467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:56.744467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:56.744467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:56.744467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:56.744467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:56.744467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:56.744467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:56.744467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:56.744467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:56.745616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:56.745616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:56.745616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:56.745616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:56.745616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:56.745616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:56.745616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:56.745616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:56.745616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:56.745616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:56.745616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:56.745616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:56.745616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:56.745616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:56.745616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:56.745616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:56.745616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:56.745616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:56.746547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:56.746547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:56.746547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:56.746547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:56.746547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:56.746547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:56.746547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:56.746547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:56.746547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 08:20:56.746547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:56.751089 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 483/963] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jun 7 08:20:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:56.752660 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09baea47-d54e-4b19-92a2-9f49f91d05c8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:56.757836 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cce77-fc23-4174-9816-5f157bb65b5e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:56.795435 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7a084-a2bf-403f-9349-d00c52fc1d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:56.796511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7a084-a2bf-403f-9349-d00c52fc1d9f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:56.796936 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7a084-a2bf-403f-9349-d00c52fc1d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:56.797505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7a084-a2bf-403f-9349-d00c52fc1d9f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:56.798644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7a084-a2bf-403f-9349-d00c52fc1d9f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:56.800180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7a084-a2bf-403f-9349-d00c52fc1d9f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:56.812760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-16c7a084-a2bf-403f-9349-d00c52fc1d9f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:20:56.812760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:56.812760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:20:56.812760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:20:56.812760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:20:56.812760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:20:56.812760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:20:56.812760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:56.812760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:20:56.812760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:56.812760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:56.812760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:56.812760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:56.812760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:56.812760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:56.812760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:56.812760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:56.814283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:56.814283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:56.814283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:56.814283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:56.814283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:56.814283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:56.814283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:20:56.814283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:20:56.814283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:20:56.814283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:20:56.814283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:56.814283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:56.814283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:56.814283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:56.814283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:56.814283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:56.815142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:56.815142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:56.815142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:56.815142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:56.815142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:56.815142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:56.815142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:56.815142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:56.815142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:56.815142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:56.815142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:20:56.815142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:20:56.815142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:20:56.815142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:56.822874 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 481/964] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jun 7 08:20:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:56.824018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7a084-a2bf-403f-9349-d00c52fc1d9f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:56.857950 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09baea47-d54e-4b19-92a2-9f49f91d05c8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:56.892839 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d2d92-10b1-4b4b-a2dd-45797c4552c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:56.893643 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d2d92-10b1-4b4b-a2dd-45797c4552c1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:56.894001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d2d92-10b1-4b4b-a2dd-45797c4552c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:56.894500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d2d92-10b1-4b4b-a2dd-45797c4552c1 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:56.895452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d2d92-10b1-4b4b-a2dd-45797c4552c1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:56.895865 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d2d92-10b1-4b4b-a2dd-45797c4552c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:56.897462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d2d92-10b1-4b4b-a2dd-45797c4552c1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:56.912367 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:56.921281 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 484/965] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Jun 7 08:20:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:56.922712 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d2d92-10b1-4b4b-a2dd-45797c4552c1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:56.934953 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7a084-a2bf-403f-9349-d00c52fc1d9f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:56.970484 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b424b-53ba-407f-923c-bb37597a90ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:56.971440 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b424b-53ba-407f-923c-bb37597a90ca None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:56.971923 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b424b-53ba-407f-923c-bb37597a90ca None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:56.972339 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b424b-53ba-407f-923c-bb37597a90ca None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:56.973213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b424b-53ba-407f-923c-bb37597a90ca None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:56.973661 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b424b-53ba-407f-923c-bb37597a90ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:56.975233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b424b-53ba-407f-923c-bb37597a90ca None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:56.985264 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 482/966] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jun 7 08:20:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:56.987118 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b424b-53ba-407f-923c-bb37597a90ca None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:56.993403 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:57.026722 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d2d92-10b1-4b4b-a2dd-45797c4552c1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:57.064537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149402b-e687-4032-b34b-1c1572c583b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:57.065417 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149402b-e687-4032-b34b-1c1572c583b5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:57.065867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149402b-e687-4032-b34b-1c1572c583b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:57.066281 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149402b-e687-4032-b34b-1c1572c583b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:57.093802 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b424b-53ba-407f-923c-bb37597a90ca None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:57.130155 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6357be-d0da-4875-b918-3a2c98dc20ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:57.130752 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6357be-d0da-4875-b918-3a2c98dc20ef None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:57.131041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6357be-d0da-4875-b918-3a2c98dc20ef None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:57.131321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6357be-d0da-4875-b918-3a2c98dc20ef None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:57.132031 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6357be-d0da-4875-b918-3a2c98dc20ef None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:57.132341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6357be-d0da-4875-b918-3a2c98dc20ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:57.133741 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6357be-d0da-4875-b918-3a2c98dc20ef None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:57.137280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149402b-e687-4032-b34b-1c1572c583b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:57.139425 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149402b-e687-4032-b34b-1c1572c583b5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:57.139758 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 483/967] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jun 7 08:20:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:57.141101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c25e9-8760-4ee6-a856-669074331e55 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:57.141391 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c25e9-8760-4ee6-a856-669074331e55 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:57.141672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c25e9-8760-4ee6-a856-669074331e55 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:57.143053 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 484/968] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:57.148055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8e961-b56d-4a30-9b76-d1cee1c05a28 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:57.148344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8e961-b56d-4a30-9b76-d1cee1c05a28 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:57.148621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8e961-b56d-4a30-9b76-d1cee1c05a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:57.165964 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1af8e961-b56d-4a30-9b76-d1cee1c05a28 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:57.167141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1af8e961-b56d-4a30-9b76-d1cee1c05a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:57.170968 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:57.201998 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:57.249712 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 485/969] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:57.257212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 485/970] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:20:56 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:20:57.257212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149402b-e687-4032-b34b-1c1572c583b5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:57.261985 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8e961-b56d-4a30-9b76-d1cee1c05a28 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:57.368949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149402b-e687-4032-b34b-1c1572c583b5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:57.374652 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8e961-b56d-4a30-9b76-d1cee1c05a28 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:57.408731 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6fa7e-ec7f-4df1-9c83-a9c1db076e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:57.409288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6fa7e-ec7f-4df1-9c83-a9c1db076e2d None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:57.409572 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6fa7e-ec7f-4df1-9c83-a9c1db076e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:57.409850 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6fa7e-ec7f-4df1-9c83-a9c1db076e2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:57.410590 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6fa7e-ec7f-4df1-9c83-a9c1db076e2d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:57.411965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6fa7e-ec7f-4df1-9c83-a9c1db076e2d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:57.412410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538cd41-7292-45cc-8d86-3c85892dcee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:57.413314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538cd41-7292-45cc-8d86-3c85892dcee8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:57.413600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538cd41-7292-45cc-8d86-3c85892dcee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:57.413869 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538cd41-7292-45cc-8d86-3c85892dcee8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:57.422601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:57.424875 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538cd41-7292-45cc-8d86-3c85892dcee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:57.426426 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538cd41-7292-45cc-8d86-3c85892dcee8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:57.443729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-4538cd41-7292-45cc-8d86-3c85892dcee8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:57.443729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:57.443729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:57.443729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:57.443729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:57.443729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:57.443729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:57.443729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:57.443729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:57.443729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:57.443729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:57.443729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:57.443729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:20:57.443729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:20:57.443729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:20:57.444812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:20:57.444812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:57.444812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:57.444812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:57.444812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:57.444812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:57.444812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:57.444812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:57.444812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:57.444812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:57.444812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:57.444812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:57.444812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:57.444812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:57.444812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:57.444812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:57.444812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:57.445887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:20:57.445887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:20:57.445887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:20:57.445887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:57.445887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 486/971] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:20:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:57.446172 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538cd41-7292-45cc-8d86-3c85892dcee8 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:57.477955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 486/972] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jun 7 08:20:57 2022] POST /identity/v3/endpoints => generated 349 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:57.479649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6fa7e-ec7f-4df1-9c83-a9c1db076e2d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:57.502120 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:57.565964 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538cd41-7292-45cc-8d86-3c85892dcee8 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:57.612252 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fec807-a0b1-4f2c-b45c-1f3307b0f5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:57.613116 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fec807-a0b1-4f2c-b45c-1f3307b0f5fd None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:57.613474 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fec807-a0b1-4f2c-b45c-1f3307b0f5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:57.613820 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fec807-a0b1-4f2c-b45c-1f3307b0f5fd None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:57.614647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fec807-a0b1-4f2c-b45c-1f3307b0f5fd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:57.616144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fec807-a0b1-4f2c-b45c-1f3307b0f5fd None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:57.626868 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6fa7e-ec7f-4df1-9c83-a9c1db076e2d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:57.665499 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83943ee7-0f48-42da-92c2-01b42d65243e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:57.666189 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83943ee7-0f48-42da-92c2-01b42d65243e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:57.666508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83943ee7-0f48-42da-92c2-01b42d65243e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:57.666747 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83943ee7-0f48-42da-92c2-01b42d65243e None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:57.667566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83943ee7-0f48-42da-92c2-01b42d65243e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:57.667874 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83943ee7-0f48-42da-92c2-01b42d65243e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:57.669364 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83943ee7-0f48-42da-92c2-01b42d65243e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:57.682503 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 487/973] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jun 7 08:20:57 2022] POST /identity/v3/services => generated 242 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:20:57.691288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 487/974] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:57.701245 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fec807-a0b1-4f2c-b45c-1f3307b0f5fd None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:57.759406 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:57.827304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fec807-a0b1-4f2c-b45c-1f3307b0f5fd None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:57.868390 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104d08f-5fe0-4962-8765-0ba514fe8bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:57.869686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104d08f-5fe0-4962-8765-0ba514fe8bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:57.870140 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104d08f-5fe0-4962-8765-0ba514fe8bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:57.870563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104d08f-5fe0-4962-8765-0ba514fe8bf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:57.879005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60118c5-3932-4a3a-af89-c0d8e56ec307 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:57.879570 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60118c5-3932-4a3a-af89-c0d8e56ec307 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:57.879570 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60118c5-3932-4a3a-af89-c0d8e56ec307 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:57.879963 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7104d08f-5fe0-4962-8765-0ba514fe8bf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:57.881664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7104d08f-5fe0-4962-8765-0ba514fe8bf3 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:57.884888 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 488/975] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:20:57.886176 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4168ef-5d81-4c9f-a974-0170641182e0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:57.886460 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4168ef-5d81-4c9f-a974-0170641182e0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:57.886734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4168ef-5d81-4c9f-a974-0170641182e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:57.898768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-7104d08f-5fe0-4962-8765-0ba514fe8bf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:57.898768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:57.898768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:57.898768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:57.898768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:57.898768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:57.898768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:57.898768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:57.898768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:57.898768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:57.898768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:57.898768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:57.898768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:20:57.898768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:20:57.898768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:20:57.899710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:20:57.899710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:57.899710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:57.899710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:57.899710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:57.899710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:57.899710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:57.899710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:57.899710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:57.899710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:57.899710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:57.899710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:57.899710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:57.899710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:57.899710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:57.899710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:57.899710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:57.900633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:20:57.900633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:20:57.900633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:20:57.900633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:20:57.900633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:20:57.900633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:20:57.900633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:20:57.900633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:57.900633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 488/976] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:20:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:57.906900 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2b4168ef-5d81-4c9f-a974-0170641182e0 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:57.907067 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104d08f-5fe0-4962-8765-0ba514fe8bf3 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:57.907187 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2b4168ef-5d81-4c9f-a974-0170641182e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:20:57.990866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:57.998531 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 489/977] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:20:58.021447 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104d08f-5fe0-4962-8765-0ba514fe8bf3 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:58.024255 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:58.028157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4168ef-5d81-4c9f-a974-0170641182e0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:58.063421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1883c-7096-4731-a8ea-9c6f7596b442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:58.064081 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1883c-7096-4731-a8ea-9c6f7596b442 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:58.064360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1883c-7096-4731-a8ea-9c6f7596b442 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:58.064630 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1883c-7096-4731-a8ea-9c6f7596b442 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:58.065437 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1883c-7096-4731-a8ea-9c6f7596b442 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:58.065740 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1883c-7096-4731-a8ea-9c6f7596b442 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:58.067410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1883c-7096-4731-a8ea-9c6f7596b442 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:58.084946 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 489/978] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:20:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:58.086479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1883c-7096-4731-a8ea-9c6f7596b442 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:58.165290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4168ef-5d81-4c9f-a974-0170641182e0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:58.204063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1883c-7096-4731-a8ea-9c6f7596b442 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:58.209021 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477b045-376a-405d-8b55-1f1f4f1a64a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:58.209701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477b045-376a-405d-8b55-1f1f4f1a64a3 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:58.209980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477b045-376a-405d-8b55-1f1f4f1a64a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:58.210250 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477b045-376a-405d-8b55-1f1f4f1a64a3 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:58.210946 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6477b045-376a-405d-8b55-1f1f4f1a64a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:58.212486 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6477b045-376a-405d-8b55-1f1f4f1a64a3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:58.240153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b13d2-c2e1-45e5-854c-5b1694213484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:58.240807 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b13d2-c2e1-45e5-854c-5b1694213484 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:58.241174 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b13d2-c2e1-45e5-854c-5b1694213484 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:58.241396 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b13d2-c2e1-45e5-854c-5b1694213484 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:58.242221 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b13d2-c2e1-45e5-854c-5b1694213484 None admin] RBAC: Adding query filter params (role.id=c782c67dcd6a46cc8c5b26a87d6d2dc3, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=bbf0ce396885472bb5d225aa009043f2) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:58.242532 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b13d2-c2e1-45e5-854c-5b1694213484 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:58.243959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b13d2-c2e1-45e5-854c-5b1694213484 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:58.250392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 490/979] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:20:58 2022] GET /identity/v3/role_assignments?user.id=bbf0ce396885472bb5d225aa009043f2&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=c782c67dcd6a46cc8c5b26a87d6d2dc3 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:20:58.253198 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.sql.core [None req-6477b045-376a-405d-8b55-1f1f4f1a64a3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 08:20:58.253198 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 07 08:20:58.253198 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [parameters: {'user_id': 'a1ebe796148e4f69acab76ad574fbcdd', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 07 08:20:58.253198 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80358) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 08:20:58.261941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-6477b045-376a-405d-8b55-1f1f4f1a64a3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 07 08:20:58.261941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:58.261941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 08:20:58.261941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 08:20:58.261941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 08:20:58.261941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 08:20:58.261941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 08:20:58.261941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 08:20:58.261941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 08:20:58.261941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application conn.query(q) Jun 07 08:20:58.261941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 08:20:58.261941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 08:20:58.261941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 08:20:58.261941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result.read() Jun 07 08:20:58.261941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 08:20:58.262986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 08:20:58.262986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 08:20:58.262986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 08:20:58.262986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 08:20:58.262986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 08:20:58.262986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 08:20:58.262986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 08:20:58.262986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 08:20:58.262986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:58.262986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 08:20:58.262986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:58.262986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:58.262986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 08:20:58.262986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 08:20:58.262986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 07 08:20:58.262986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 07 08:20:58.262986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 08:20:58.262986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application next(self.gen) Jun 07 08:20:58.263965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 08:20:58.263965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application yield resource Jun 07 08:20:58.263965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 08:20:58.263965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application next(self.gen) Jun 07 08:20:58.263965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 08:20:58.263965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 08:20:58.263965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 08:20:58.263965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 08:20:58.263965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 08:20:58.263965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise self.value Jun 07 08:20:58.263965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 08:20:58.263965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 08:20:58.263965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 08:20:58.263965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application session.commit() Jun 07 08:20:58.263965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 08:20:58.263965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 08:20:58.263965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 08:20:58.264866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 08:20:58.264866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 08:20:58.264866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.session.flush() Jun 07 08:20:58.264866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 08:20:58.264866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 08:20:58.264866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 08:20:58.264866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 08:20:58.264866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 08:20:58.264866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application compat.raise_( Jun 07 08:20:58.264866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 08:20:58.264866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception Jun 07 08:20:58.264866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 08:20:58.264866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 08:20:58.264866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 08:20:58.264866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 08:20:58.264866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 08:20:58.264866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 08:20:58.271337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 08:20:58.271337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 08:20:58.271337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 07 08:20:58.271337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 07 08:20:58.271337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 08:20:58.271337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 08:20:58.271337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 08:20:58.271337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 08:20:58.271337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 08:20:58.271337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 08:20:58.271337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 08:20:58.271337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 08:20:58.271337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 08:20:58.271337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 08:20:58.271337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 08:20:58.271337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception Jun 07 08:20:58.272215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 08:20:58.272215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 08:20:58.272215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 08:20:58.272215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 08:20:58.272215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 08:20:58.272215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 08:20:58.272215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 08:20:58.272215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application conn.query(q) Jun 07 08:20:58.272215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 08:20:58.272215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 08:20:58.272215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 08:20:58.272215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result.read() Jun 07 08:20:58.272215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 08:20:58.272215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 08:20:58.272215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 08:20:58.272215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 08:20:58.273096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 08:20:58.273096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 08:20:58.273096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 08:20:58.273096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 08:20:58.273096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 08:20:58.273096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 07 08:20:58.273096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a1ebe796148e4f69acab76ad574fbcdd', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 07 08:20:58.273096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 08:20:58.273096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:58.273096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:20:58.273096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:58.273096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:58.273096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:58.273096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:58.273096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:58.274242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:58.274242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:58.274242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:58.274242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:58.274242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:58.274242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:58.274242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:58.274242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 07 08:20:58.274242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 07 08:20:58.274242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:58.274242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:58.274242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:58.274242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:58.274242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:58.274242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:58.274242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 07 08:20:58.274242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 07 08:20:58.275333 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 07 08:20:58.275333 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 07 08:20:58.275333 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 08:20:58.275333 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 08:20:58.275333 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 07 08:20:58.275333 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:20:58.275333 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 490/980] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Jun 7 08:20:58 2022] POST /identity/v3/users => generated 160 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:20:58.276902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b13d2-c2e1-45e5-854c-5b1694213484 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:58.314690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:58.399262 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b13d2-c2e1-45e5-854c-5b1694213484 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:58.438114 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ce020-98e9-41b4-af9f-c81b7b239c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:58.438926 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ce020-98e9-41b4-af9f-c81b7b239c63 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:58.439203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ce020-98e9-41b4-af9f-c81b7b239c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:58.439470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ce020-98e9-41b4-af9f-c81b7b239c63 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:58.448289 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ce020-98e9-41b4-af9f-c81b7b239c63 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:58.449922 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ce020-98e9-41b4-af9f-c81b7b239c63 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:58.464904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-3d9ce020-98e9-41b4-af9f-c81b7b239c63 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 08:20:58.464904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:20:58.464904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:20:58.464904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:20:58.464904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:20:58.464904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:20:58.464904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:20:58.464904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:20:58.464904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:20:58.464904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:20:58.464904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:20:58.464904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:20:58.464904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:20:58.464904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:20:58.464904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:20:58.464904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:20:58.466463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:20:58.466463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:20:58.466463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:20:58.466463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:58.466463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:20:58.466463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:20:58.466463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:20:58.466463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:20:58.466463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:20:58.466463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:20:58.466463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:20:58.466463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:20:58.466463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:20:58.466463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:20:58.466463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:20:58.466463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:20:58.466463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:20:58.466463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:20:58.467506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:20:58.467506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:20:58.467506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:20:58.467506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:20:58.467506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:20:58.467506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:20:58.467506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:20:58.467506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:20:58.467506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 08:20:58.467506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:20:58.467506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 491/981] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:20:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:20:58.474973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477b045-376a-405d-8b55-1f1f4f1a64a3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:20:58.541512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:20:58.581619 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477b045-376a-405d-8b55-1f1f4f1a64a3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:20:58.618043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8b8c5-0940-4f54-8fa1-bfe332ffa579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:20:58.618711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8b8c5-0940-4f54-8fa1-bfe332ffa579 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:58.618986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8b8c5-0940-4f54-8fa1-bfe332ffa579 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:58.619258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8b8c5-0940-4f54-8fa1-bfe332ffa579 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:58.620049 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8b8c5-0940-4f54-8fa1-bfe332ffa579 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:20:58.620350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8b8c5-0940-4f54-8fa1-bfe332ffa579 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:20:58.622052 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8b8c5-0940-4f54-8fa1-bfe332ffa579 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:20:58.660196 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 491/982] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Jun 7 08:20:58 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:20:59.908435 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0350521-6c63-4334-a509-b2327c3b83f1 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:59.908959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0350521-6c63-4334-a509-b2327c3b83f1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:59.908959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0350521-6c63-4334-a509-b2327c3b83f1 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:59.910391 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 492/983] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:20:59.915567 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15b4b6-3534-47b2-9f35-e451fb207355 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:20:59.915950 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15b4b6-3534-47b2-9f35-e451fb207355 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:20:59.916141 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15b4b6-3534-47b2-9f35-e451fb207355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:20:59.933242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bb15b4b6-3534-47b2-9f35-e451fb207355 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:20:59.935283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bb15b4b6-3534-47b2-9f35-e451fb207355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:00.002697 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 492/984] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:20:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:00.024821 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0350521-6c63-4334-a509-b2327c3b83f1 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:00.036399 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4f2e4-828b-447c-81b6-e11c5e34d421 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.036692 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4f2e4-828b-447c-81b6-e11c5e34d421 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.036981 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4f2e4-828b-447c-81b6-e11c5e34d421 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.038374 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 493/985] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:21:00.043716 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8104d7-cbc3-48e3-b4d9-233f5a75cde4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.044003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8104d7-cbc3-48e3-b4d9-233f5a75cde4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.044280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8104d7-cbc3-48e3-b4d9-233f5a75cde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.061710 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-df8104d7-cbc3-48e3-b4d9-233f5a75cde4 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:00.062889 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-df8104d7-cbc3-48e3-b4d9-233f5a75cde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:00.110877 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:00.120232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:00.132663 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 494/986] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:21:00.133603 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0350521-6c63-4334-a509-b2327c3b83f1 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:00.154818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8104d7-cbc3-48e3-b4d9-233f5a75cde4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:00.167711 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3d47a-410b-4303-b98f-7b275a108143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:00.168527 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3d47a-410b-4303-b98f-7b275a108143 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.169365 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3d47a-410b-4303-b98f-7b275a108143 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.169638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3d47a-410b-4303-b98f-7b275a108143 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.170290 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3d47a-410b-4303-b98f-7b275a108143 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:00.171725 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3d47a-410b-4303-b98f-7b275a108143 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:00.185897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-dea3d47a-410b-4303-b98f-7b275a108143 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:21:00.185897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:00.185897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:21:00.185897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:21:00.185897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:21:00.185897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:21:00.185897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:21:00.185897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:21:00.185897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:21:00.185897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:00.185897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:21:00.185897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:00.185897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:00.185897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:00.185897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:00.185897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:00.185897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:00.187382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:00.187382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:00.187382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:00.187382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:00.187382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:00.187382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:00.187382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:21:00.187382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:21:00.187382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:21:00.187382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:21:00.187382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:00.187382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:00.187382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:00.187382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:00.187382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:00.187382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:00.188449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:00.188449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:00.188449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:00.188449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:00.188449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:00.188449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:00.188449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:00.188449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:00.188449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:00.188449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:00.188449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:21:00.188449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:21:00.188449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:21:00.188449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:00.188449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 493/987] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jun 7 08:21:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:00.192314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3d47a-410b-4303-b98f-7b275a108143 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:00.273137 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8104d7-cbc3-48e3-b4d9-233f5a75cde4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:00.298061 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3d47a-410b-4303-b98f-7b275a108143 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:00.313066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276217b-c38e-41b5-b3c3-f90563db2ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:00.313879 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276217b-c38e-41b5-b3c3-f90563db2ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.314156 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276217b-c38e-41b5-b3c3-f90563db2ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.314457 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276217b-c38e-41b5-b3c3-f90563db2ac6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.315087 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276217b-c38e-41b5-b3c3-f90563db2ac6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:00.316509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276217b-c38e-41b5-b3c3-f90563db2ac6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:00.330018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-0276217b-c38e-41b5-b3c3-f90563db2ac6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 08:21:00.330018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:00.330018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:21:00.330018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:21:00.330018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:21:00.330018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:21:00.330018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:21:00.330018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:21:00.330018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 08:21:00.330018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:00.330018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:21:00.330018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:00.330018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:00.330018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:00.330018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:00.330018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:00.330018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:00.331107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:00.331107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:00.331107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:00.331107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:00.331107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:00.331107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:00.331107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:21:00.331107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:21:00.331107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:21:00.331107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:21:00.331107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:00.331107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:00.331107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:00.331107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:00.331107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:00.331107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:00.331942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:00.331942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:00.331942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:00.331942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:00.331942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:00.331942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:00.331942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:00.331942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:00.331942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:00.331942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:00.331942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:21:00.331942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:21:00.331942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 08:21:00.331942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:00.331942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 495/988] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:21:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:00.336983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276217b-c38e-41b5-b3c3-f90563db2ac6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:00.339151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18719e7f-1e23-488a-a9b6-77a2016e2ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:00.339820 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18719e7f-1e23-488a-a9b6-77a2016e2ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.340111 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18719e7f-1e23-488a-a9b6-77a2016e2ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.341201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18719e7f-1e23-488a-a9b6-77a2016e2ba3 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.341201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18719e7f-1e23-488a-a9b6-77a2016e2ba3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:00.341461 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18719e7f-1e23-488a-a9b6-77a2016e2ba3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:00.342863 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18719e7f-1e23-488a-a9b6-77a2016e2ba3 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:00.347905 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 494/989] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jun 7 08:21:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:00.353336 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18719e7f-1e23-488a-a9b6-77a2016e2ba3 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:00.372887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:00.388015 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:00.447675 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276217b-c38e-41b5-b3c3-f90563db2ac6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:00.457960 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18719e7f-1e23-488a-a9b6-77a2016e2ba3 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:00.484746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c4e93-7fac-4f99-8d7b-335496f2b365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:00.485837 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c4e93-7fac-4f99-8d7b-335496f2b365 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.486300 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c4e93-7fac-4f99-8d7b-335496f2b365 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.486803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c4e93-7fac-4f99-8d7b-335496f2b365 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.487680 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c4e93-7fac-4f99-8d7b-335496f2b365 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:00.488229 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c4e93-7fac-4f99-8d7b-335496f2b365 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:00.489937 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c4e93-7fac-4f99-8d7b-335496f2b365 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:00.491328 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd1fcd-a294-440b-a362-351a09061039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:00.491989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd1fcd-a294-440b-a362-351a09061039 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.492272 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd1fcd-a294-440b-a362-351a09061039 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.492546 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd1fcd-a294-440b-a362-351a09061039 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.493274 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd1fcd-a294-440b-a362-351a09061039 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:00.493579 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd1fcd-a294-440b-a362-351a09061039 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:00.494931 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd1fcd-a294-440b-a362-351a09061039 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:00.496835 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 496/990] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jun 7 08:21:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:00.502320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c4e93-7fac-4f99-8d7b-335496f2b365 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:00.509238 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 495/991] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jun 7 08:21:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:00.510483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd1fcd-a294-440b-a362-351a09061039 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:00.592656 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:00.612601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd1fcd-a294-440b-a362-351a09061039 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:00.613062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c4e93-7fac-4f99-8d7b-335496f2b365 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:00.622282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:00.646504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc386bb-aee8-476d-a97a-e53eec6608bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:00.646951 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bd07a-2894-40c9-a2f1-44bfc1d6b149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:00.647156 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc386bb-aee8-476d-a97a-e53eec6608bf None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.647440 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc386bb-aee8-476d-a97a-e53eec6608bf None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.647713 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc386bb-aee8-476d-a97a-e53eec6608bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.647925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bd07a-2894-40c9-a2f1-44bfc1d6b149 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.648212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bd07a-2894-40c9-a2f1-44bfc1d6b149 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.648493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bd07a-2894-40c9-a2f1-44bfc1d6b149 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.648581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc386bb-aee8-476d-a97a-e53eec6608bf None admin] RBAC: Adding query filter params (interface=admin, service_id=be4491ad274e4214be45b5f282d990b8, region_id=RegionOne) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:00.648801 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc386bb-aee8-476d-a97a-e53eec6608bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:00.649223 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547bd07a-2894-40c9-a2f1-44bfc1d6b149 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:00.649533 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547bd07a-2894-40c9-a2f1-44bfc1d6b149 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:00.650203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc386bb-aee8-476d-a97a-e53eec6608bf None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:00.650881 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547bd07a-2894-40c9-a2f1-44bfc1d6b149 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:00.655660 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 496/992] 10.222.0.17 () {56 vars in 1249 bytes} [Tue Jun 7 08:21:00 2022] GET /identity/v3/endpoints?service_id=be4491ad274e4214be45b5f282d990b8&interface=admin®ion_id=RegionOne => generated 193 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:00.661063 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 497/993] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jun 7 08:21:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:00.661200 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22178675-8647-4f9b-b607-34db13704c6a None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.661200 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22178675-8647-4f9b-b607-34db13704c6a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.661200 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22178675-8647-4f9b-b607-34db13704c6a None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.661200 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 497/994] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:00.661466 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bd07a-2894-40c9-a2f1-44bfc1d6b149 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:00.662203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22178675-8647-4f9b-b607-34db13704c6a None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:00.756886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bd07a-2894-40c9-a2f1-44bfc1d6b149 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:00.759350 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22178675-8647-4f9b-b607-34db13704c6a None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:00.789025 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5af62f-e957-4e02-86cb-24e4db029642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:00.789692 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5af62f-e957-4e02-86cb-24e4db029642 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.789974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5af62f-e957-4e02-86cb-24e4db029642 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.790252 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5af62f-e957-4e02-86cb-24e4db029642 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.791271 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5af62f-e957-4e02-86cb-24e4db029642 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:00.792529 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d8817-bce3-439b-9871-4779c610c5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:00.792667 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5af62f-e957-4e02-86cb-24e4db029642 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:00.793204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d8817-bce3-439b-9871-4779c610c5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.793626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d8817-bce3-439b-9871-4779c610c5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.794359 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d8817-bce3-439b-9871-4779c610c5f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.795144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d8817-bce3-439b-9871-4779c610c5f8 None admin] RBAC: Adding query filter params (interface=public, service_id=3632ccfbbbed4828ba458cb3371ea198, region_id=RegionOne) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:00.795461 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d8817-bce3-439b-9871-4779c610c5f8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:00.796828 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d8817-bce3-439b-9871-4779c610c5f8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:00.799633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 498/995] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Jun 7 08:21:00 2022] GET /identity/v3/services => generated 1727 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:21:00.800781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40871a1-324b-41e5-939f-beab7628c63e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.801093 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40871a1-324b-41e5-939f-beab7628c63e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.801368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40871a1-324b-41e5-939f-beab7628c63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.807583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 498/996] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Jun 7 08:21:00 2022] GET /identity/v3/endpoints?service_id=3632ccfbbbed4828ba458cb3371ea198&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:00.808861 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d8817-bce3-439b-9871-4779c610c5f8 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:00.821572 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:00.823456 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a40871a1-324b-41e5-939f-beab7628c63e None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:00.824959 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a40871a1-324b-41e5-939f-beab7628c63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:00.858828 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:00.908266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 499/997] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:00.925851 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d8817-bce3-439b-9871-4779c610c5f8 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:00.930834 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40871a1-324b-41e5-939f-beab7628c63e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:00.967870 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1aab8-ef51-4299-aa2a-92200da2fd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:00.967870 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1aab8-ef51-4299-aa2a-92200da2fd23 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:00.968200 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1aab8-ef51-4299-aa2a-92200da2fd23 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:00.968419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1aab8-ef51-4299-aa2a-92200da2fd23 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:00.969098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1aab8-ef51-4299-aa2a-92200da2fd23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:00.970515 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1aab8-ef51-4299-aa2a-92200da2fd23 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:00.977753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 499/998] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Jun 7 08:21:00 2022] GET /identity/v3/services => generated 1727 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:21:01.054525 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40871a1-324b-41e5-939f-beab7628c63e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:01.092939 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b0fef-169d-4dca-8cec-b3ef512b78cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:01.093777 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b0fef-169d-4dca-8cec-b3ef512b78cd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:01.094059 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b0fef-169d-4dca-8cec-b3ef512b78cd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:01.094332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b0fef-169d-4dca-8cec-b3ef512b78cd None admin] PATH_INFO: `/v3/services/image` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:01.094990 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b0fef-169d-4dca-8cec-b3ef512b78cd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:01.096372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b0fef-169d-4dca-8cec-b3ef512b78cd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:01.109188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-7e5b0fef-169d-4dca-8cec-b3ef512b78cd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 08:21:01.109188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:01.109188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:21:01.109188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:21:01.109188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:21:01.109188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:21:01.109188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:21:01.109188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:21:01.109188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 08:21:01.109188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:01.109188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:21:01.109188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:01.109188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:01.109188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:01.109188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:01.109188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:01.109188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:01.110383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:01.110383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:01.110383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:01.110383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:01.110383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:01.110383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:01.110383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:21:01.110383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:21:01.110383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:21:01.110383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:21:01.110383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:01.110383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:01.110383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:01.110383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:01.110383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:01.110383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:01.111624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:01.111624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:01.111624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:01.111624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:01.111624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:01.111624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:01.111624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:01.111624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:01.111624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:01.111624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:01.111624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:21:01.111624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:21:01.111624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 08:21:01.111624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:01.111624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 500/999] 10.222.0.17 () {56 vars in 1105 bytes} [Tue Jun 7 08:21:00 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:01.116625 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1aab8-ef51-4299-aa2a-92200da2fd23 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:01.197588 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:01.228527 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1aab8-ef51-4299-aa2a-92200da2fd23 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:01.261992 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc5246-5a02-4a39-b0c8-ec2040204bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:01.263305 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc5246-5a02-4a39-b0c8-ec2040204bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:01.263581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc5246-5a02-4a39-b0c8-ec2040204bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:01.263854 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc5246-5a02-4a39-b0c8-ec2040204bd8 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:01.264558 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc5246-5a02-4a39-b0c8-ec2040204bd8 None admin] RBAC: Adding query filter params (name=image) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:01.264860 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc5246-5a02-4a39-b0c8-ec2040204bd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:01.266251 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc5246-5a02-4a39-b0c8-ec2040204bd8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:01.271034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 500/1000] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:01.276886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b0fef-169d-4dca-8cec-b3ef512b78cd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:01.307667 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:01.377449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b0fef-169d-4dca-8cec-b3ef512b78cd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:01.410916 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e64120-0b38-4f14-afa8-9b0719dfaa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:01.411575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e64120-0b38-4f14-afa8-9b0719dfaa90 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:01.411853 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e64120-0b38-4f14-afa8-9b0719dfaa90 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:01.412131 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e64120-0b38-4f14-afa8-9b0719dfaa90 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:01.412811 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e64120-0b38-4f14-afa8-9b0719dfaa90 None admin] RBAC: Adding query filter params (type=image) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:01.413143 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e64120-0b38-4f14-afa8-9b0719dfaa90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:01.414509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e64120-0b38-4f14-afa8-9b0719dfaa90 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:01.419135 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 501/1001] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:01 2022] GET /identity/v3/services?type=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:03.080120 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf0155-50cb-4757-93ec-c683f75d5827 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.080650 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf0155-50cb-4757-93ec-c683f75d5827 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.080650 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf0155-50cb-4757-93ec-c683f75d5827 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.082191 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 501/1002] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:03.087199 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9a216-3043-435a-a6d4-8fdee65711de None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.087580 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9a216-3043-435a-a6d4-8fdee65711de None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.087752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9a216-3043-435a-a6d4-8fdee65711de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.104547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-75d9a216-3043-435a-a6d4-8fdee65711de None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:03.105725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-75d9a216-3043-435a-a6d4-8fdee65711de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:03.181156 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 502/1003] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:03.197300 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf0155-50cb-4757-93ec-c683f75d5827 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:03.250789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22d224-cda4-45fb-9f5e-72866dca9e86 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.251128 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22d224-cda4-45fb-9f5e-72866dca9e86 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.251360 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22d224-cda4-45fb-9f5e-72866dca9e86 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.252738 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 503/1004] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:03.257719 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8c99a-1601-40a3-84e0-e3321a1313c4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.258011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8c99a-1601-40a3-84e0-e3321a1313c4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.258294 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8c99a-1601-40a3-84e0-e3321a1313c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.276804 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5fe8c99a-1601-40a3-84e0-e3321a1313c4 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:03.278722 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5fe8c99a-1601-40a3-84e0-e3321a1313c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:03.297660 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:03.303863 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf0155-50cb-4757-93ec-c683f75d5827 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:03.342885 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f13498-0118-40b8-9708-c376a900560c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:03.343595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f13498-0118-40b8-9708-c376a900560c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.343873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f13498-0118-40b8-9708-c376a900560c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.344144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f13498-0118-40b8-9708-c376a900560c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.344788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f13498-0118-40b8-9708-c376a900560c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:03.346182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f13498-0118-40b8-9708-c376a900560c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:03.348726 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:03.358711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 504/1005] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:21:03.363615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-b2f13498-0118-40b8-9708-c376a900560c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:21:03.363615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:03.363615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:21:03.363615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:21:03.363615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:21:03.363615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:21:03.363615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:21:03.363615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:21:03.363615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:21:03.363615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:03.363615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:21:03.363615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:03.363615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:03.363615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:03.363615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:03.363615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:03.363615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:03.364672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:03.364672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:03.364672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:03.364672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:03.364672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:03.364672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:03.364672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:21:03.364672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:21:03.364672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:21:03.364672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:21:03.364672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:03.364672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:03.364672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:03.364672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:03.364672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:03.364672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:03.365624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:03.365624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:03.365624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:03.365624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:03.365624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:03.365624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:03.365624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:03.365624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:03.365624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:03.365624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:03.365624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:21:03.365624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:21:03.365624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 08:21:03.365624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:03.365624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 502/1006] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Jun 7 08:21:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:03.367600 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe8c99a-1601-40a3-84e0-e3321a1313c4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:03.382837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f13498-0118-40b8-9708-c376a900560c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:03.480068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe8c99a-1601-40a3-84e0-e3321a1313c4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:03.487067 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f13498-0118-40b8-9708-c376a900560c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:03.516289 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e6338-90d7-4c5b-a0b9-3a940920f9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:03.517316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e6338-90d7-4c5b-a0b9-3a940920f9fc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.517746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e6338-90d7-4c5b-a0b9-3a940920f9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.518271 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e6338-90d7-4c5b-a0b9-3a940920f9fc None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.519275 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e6338-90d7-4c5b-a0b9-3a940920f9fc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:03.519819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e6338-90d7-4c5b-a0b9-3a940920f9fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:03.521538 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab187cc-1d91-4b1c-b050-7e37e11bf46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:03.521816 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e6338-90d7-4c5b-a0b9-3a940920f9fc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:03.522325 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab187cc-1d91-4b1c-b050-7e37e11bf46b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.522604 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab187cc-1d91-4b1c-b050-7e37e11bf46b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.522876 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab187cc-1d91-4b1c-b050-7e37e11bf46b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.523516 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab187cc-1d91-4b1c-b050-7e37e11bf46b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:03.524861 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab187cc-1d91-4b1c-b050-7e37e11bf46b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:03.528151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 505/1007] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jun 7 08:21:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:03.533600 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e6338-90d7-4c5b-a0b9-3a940920f9fc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:03.541490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-dab187cc-1d91-4b1c-b050-7e37e11bf46b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 08:21:03.541490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:03.541490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:21:03.541490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:21:03.541490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:21:03.541490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:21:03.541490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:21:03.541490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:21:03.541490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 08:21:03.541490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:03.541490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:21:03.541490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:03.541490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:03.541490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:03.541490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:03.541490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:03.541490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:03.542472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:03.542472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:03.542472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:03.542472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:03.542472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:03.542472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:03.542472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:21:03.542472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:21:03.542472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:21:03.542472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:21:03.542472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:03.542472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:03.542472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:03.542472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:03.542472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:03.542472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:03.543309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:03.543309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:03.543309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:03.543309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:03.543309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:03.543309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:03.543309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:03.543309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:03.543309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:03.543309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:03.543309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:21:03.543309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:21:03.543309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 08:21:03.543309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:03.543309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 503/1008] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:21:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:03.548729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab187cc-1d91-4b1c-b050-7e37e11bf46b None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:03.555084 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:03.611800 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:03.639410 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e6338-90d7-4c5b-a0b9-3a940920f9fc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:03.660130 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab187cc-1d91-4b1c-b050-7e37e11bf46b None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:03.673621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69faeed4-c635-4ed7-acde-7e6065242a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:03.674273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69faeed4-c635-4ed7-acde-7e6065242a3d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.674558 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69faeed4-c635-4ed7-acde-7e6065242a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.674824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69faeed4-c635-4ed7-acde-7e6065242a3d None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.675513 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69faeed4-c635-4ed7-acde-7e6065242a3d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:03.675817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69faeed4-c635-4ed7-acde-7e6065242a3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:03.677368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69faeed4-c635-4ed7-acde-7e6065242a3d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:03.682009 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 506/1009] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Jun 7 08:21:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:03.683187 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b0e32-c0c1-4c42-b4f6-8a89b65603db None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.683468 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b0e32-c0c1-4c42-b4f6-8a89b65603db None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.683739 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b0e32-c0c1-4c42-b4f6-8a89b65603db None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.689044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 507/1010] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:03.689044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b0e32-c0c1-4c42-b4f6-8a89b65603db None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:03.698463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97951d-99f2-429b-a2fc-73b164e38d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:03.699127 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97951d-99f2-429b-a2fc-73b164e38d56 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.699411 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97951d-99f2-429b-a2fc-73b164e38d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.699685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97951d-99f2-429b-a2fc-73b164e38d56 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.700957 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97951d-99f2-429b-a2fc-73b164e38d56 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:03.701306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97951d-99f2-429b-a2fc-73b164e38d56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:03.702672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97951d-99f2-429b-a2fc-73b164e38d56 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:03.707350 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 504/1011] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jun 7 08:21:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:03.708502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d9624-5655-4698-b455-621ebc608bf9 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.709181 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d9624-5655-4698-b455-621ebc608bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.709181 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d9624-5655-4698-b455-621ebc608bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.724963 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-fe7d9624-5655-4698-b455-621ebc608bf9 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:03.726188 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-fe7d9624-5655-4698-b455-621ebc608bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:03.769773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:03.788389 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b0e32-c0c1-4c42-b4f6-8a89b65603db None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:03.792583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 505/1012] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:03.793435 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d9624-5655-4698-b455-621ebc608bf9 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:03.825505 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23720bc8-5cd8-46bf-8d14-4d7d6396c905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:03.826162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23720bc8-5cd8-46bf-8d14-4d7d6396c905 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.826543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23720bc8-5cd8-46bf-8d14-4d7d6396c905 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.826711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23720bc8-5cd8-46bf-8d14-4d7d6396c905 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.827440 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23720bc8-5cd8-46bf-8d14-4d7d6396c905 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:03.828795 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23720bc8-5cd8-46bf-8d14-4d7d6396c905 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:03.849475 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:03.889274 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d9624-5655-4698-b455-621ebc608bf9 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:03.898361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 508/1013] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jun 7 08:21:03 2022] POST /identity/v3/endpoints => generated 325 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:21:03.899153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23720bc8-5cd8-46bf-8d14-4d7d6396c905 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:03.945273 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807c1d1-50f5-4033-91ea-34e30e742e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:03.946047 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807c1d1-50f5-4033-91ea-34e30e742e50 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:03.946330 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807c1d1-50f5-4033-91ea-34e30e742e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:03.946603 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807c1d1-50f5-4033-91ea-34e30e742e50 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:03.947490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807c1d1-50f5-4033-91ea-34e30e742e50 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:03.947599 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807c1d1-50f5-4033-91ea-34e30e742e50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:03.949311 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807c1d1-50f5-4033-91ea-34e30e742e50 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:03.959324 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 506/1014] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Jun 7 08:21:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:03.960588 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807c1d1-50f5-4033-91ea-34e30e742e50 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:04.022570 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23720bc8-5cd8-46bf-8d14-4d7d6396c905 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:04.062696 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666405c0-7e02-4449-a3be-a8a5b0b6c7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:04.063908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666405c0-7e02-4449-a3be-a8a5b0b6c7bf None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:04.064187 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666405c0-7e02-4449-a3be-a8a5b0b6c7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:04.064460 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666405c0-7e02-4449-a3be-a8a5b0b6c7bf None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:04.065275 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666405c0-7e02-4449-a3be-a8a5b0b6c7bf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:04.067020 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666405c0-7e02-4449-a3be-a8a5b0b6c7bf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:04.072613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807c1d1-50f5-4033-91ea-34e30e742e50 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:04.090548 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:04.107726 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c710c-8d9e-4958-a38c-6f8755f5a778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:04.108385 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c710c-8d9e-4958-a38c-6f8755f5a778 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:04.108677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c710c-8d9e-4958-a38c-6f8755f5a778 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:04.108957 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c710c-8d9e-4958-a38c-6f8755f5a778 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:04.109783 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c710c-8d9e-4958-a38c-6f8755f5a778 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:04.114275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c710c-8d9e-4958-a38c-6f8755f5a778 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:04.167486 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 509/1015] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jun 7 08:21:03 2022] POST /identity/v3/services => generated 242 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:21:04.198256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 507/1016] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jun 7 08:21:03 2022] POST /identity/v3/endpoints => generated 334 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:21:06.158668 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9b8f5-5af9-47f1-9084-ba8aed079d16 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:06.159263 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9b8f5-5af9-47f1-9084-ba8aed079d16 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:06.159263 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9b8f5-5af9-47f1-9084-ba8aed079d16 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:06.160720 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 510/1017] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:06.165716 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b43b69-5e97-4bd7-849b-96b81d2ec4f4 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:06.166100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b43b69-5e97-4bd7-849b-96b81d2ec4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:06.166286 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b43b69-5e97-4bd7-849b-96b81d2ec4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:06.184815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-39b43b69-5e97-4bd7-849b-96b81d2ec4f4 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:06.186049 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-39b43b69-5e97-4bd7-849b-96b81d2ec4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:06.264222 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 508/1018] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:06.287597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9b8f5-5af9-47f1-9084-ba8aed079d16 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:06.290482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:06.391932 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9b8f5-5af9-47f1-9084-ba8aed079d16 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:06.427585 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d04f9-b0df-4e37-84ae-76eb388ce88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:06.428418 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d04f9-b0df-4e37-84ae-76eb388ce88a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:06.428710 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d04f9-b0df-4e37-84ae-76eb388ce88a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:06.428986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d04f9-b0df-4e37-84ae-76eb388ce88a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:06.430190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d04f9-b0df-4e37-84ae-76eb388ce88a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:06.431798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d04f9-b0df-4e37-84ae-76eb388ce88a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:06.433348 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf08b1-e88e-44f5-833b-d04ed8658a9e None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:06.438622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf08b1-e88e-44f5-833b-d04ed8658a9e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:06.438755 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 511/1019] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:06.438845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf08b1-e88e-44f5-833b-d04ed8658a9e None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:06.438845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 509/1020] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:06.440438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d04f9-b0df-4e37-84ae-76eb388ce88a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:06.440802 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65981a-3f69-4211-a867-0b4dd6fc8fd3 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:06.441178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65981a-3f69-4211-a867-0b4dd6fc8fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:06.441545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65981a-3f69-4211-a867-0b4dd6fc8fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:06.458198 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9d65981a-3f69-4211-a867-0b4dd6fc8fd3 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:06.459368 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9d65981a-3f69-4211-a867-0b4dd6fc8fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:06.512930 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:06.529054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 510/1021] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:21:06.546286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d04f9-b0df-4e37-84ae-76eb388ce88a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:06.551396 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65981a-3f69-4211-a867-0b4dd6fc8fd3 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:06.580446 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4255bd-896d-48e8-bf4d-015385ede074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:06.581009 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:06.581138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4255bd-896d-48e8-bf4d-015385ede074 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:06.581635 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4255bd-896d-48e8-bf4d-015385ede074 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:06.581635 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4255bd-896d-48e8-bf4d-015385ede074 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:06.582344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4255bd-896d-48e8-bf4d-015385ede074 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:06.583765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4255bd-896d-48e8-bf4d-015385ede074 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:06.647745 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65981a-3f69-4211-a867-0b4dd6fc8fd3 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:06.665229 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 512/1022] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jun 7 08:21:06 2022] POST /identity/v3/projects => generated 311 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:21:06.670638 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013678fd-288b-4e46-b4f3-cb3c309a1721 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:06.670926 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013678fd-288b-4e46-b4f3-cb3c309a1721 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:06.671207 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013678fd-288b-4e46-b4f3-cb3c309a1721 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:06.672706 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 513/1023] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:06.679179 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf287e-bb61-4ba1-94df-e405f2699f03 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:06.679362 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf287e-bb61-4ba1-94df-e405f2699f03 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:06.679634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf287e-bb61-4ba1-94df-e405f2699f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:06.683808 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6006b2-00e3-4f3b-92d6-914ae3b7fb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:06.684663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6006b2-00e3-4f3b-92d6-914ae3b7fb9e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:06.684938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6006b2-00e3-4f3b-92d6-914ae3b7fb9e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:06.685376 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6006b2-00e3-4f3b-92d6-914ae3b7fb9e None admin] PATH_INFO: `/v3/services/image` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:06.686134 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6006b2-00e3-4f3b-92d6-914ae3b7fb9e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:06.688795 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6006b2-00e3-4f3b-92d6-914ae3b7fb9e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:06.698064 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d2bf287e-bb61-4ba1-94df-e405f2699f03 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:06.699627 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d2bf287e-bb61-4ba1-94df-e405f2699f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:06.701792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-af6006b2-00e3-4f3b-92d6-914ae3b7fb9e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 08:21:06.701792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:06.701792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:21:06.701792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:21:06.701792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:21:06.701792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:21:06.701792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:21:06.701792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:21:06.701792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 08:21:06.701792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:06.701792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:21:06.701792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:06.701792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:06.701792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:06.701792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:06.701792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:06.701792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:06.703085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:06.703085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:06.703085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:06.703085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:06.703085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:06.703085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:06.703085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:21:06.703085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:21:06.703085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:21:06.703085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:21:06.703085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:06.703085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:06.703085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:06.703085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:06.703085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:06.703085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:06.704234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:06.704234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:06.704234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:06.704234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:06.704234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:06.704234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:06.704234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:06.704234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:06.704234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:06.704234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:06.704234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:21:06.704234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:21:06.704234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 08:21:06.704234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:06.704234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 511/1024] 10.222.0.17 () {56 vars in 1105 bytes} [Tue Jun 7 08:21:06 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:06.709514 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6006b2-00e3-4f3b-92d6-914ae3b7fb9e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:06.780565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 514/1025] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:21:06.805284 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf287e-bb61-4ba1-94df-e405f2699f03 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:06.815834 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:06.823278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6006b2-00e3-4f3b-92d6-914ae3b7fb9e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:06.861800 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6304233-b408-412c-afbb-19480fb2c6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:06.862299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:06.862914 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6304233-b408-412c-afbb-19480fb2c6ee None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:06.863190 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6304233-b408-412c-afbb-19480fb2c6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:06.863465 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6304233-b408-412c-afbb-19480fb2c6ee None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:06.864330 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6304233-b408-412c-afbb-19480fb2c6ee None admin] RBAC: Adding query filter params (name=image) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:06.864636 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6304233-b408-412c-afbb-19480fb2c6ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:06.866120 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6304233-b408-412c-afbb-19480fb2c6ee None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:06.871163 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 512/1026] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:06.877240 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6304233-b408-412c-afbb-19480fb2c6ee None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:06.914963 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf287e-bb61-4ba1-94df-e405f2699f03 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:06.953540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24994e3-a4e4-4e6d-b063-faa70337d67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:06.954286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24994e3-a4e4-4e6d-b063-faa70337d67b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:06.954626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24994e3-a4e4-4e6d-b063-faa70337d67b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:06.954965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24994e3-a4e4-4e6d-b063-faa70337d67b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:06.966087 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24994e3-a4e4-4e6d-b063-faa70337d67b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:06.974180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-c24994e3-a4e4-4e6d-b063-faa70337d67b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:21:06.974180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:06.974180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:06.974180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:06.974180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:06.974180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:06.974180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:06.974180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:06.974180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:06.974180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:06.974180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:06.974180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:06.974180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:06.974180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:06.974180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:06.974180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:06.975285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:06.975285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:06.975285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:06.975285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:06.975285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:06.975285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:06.975285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:06.975285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:06.975285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:06.975285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:06.975285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:06.975285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:06.975285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:06.975285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:06.975285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:06.975285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:06.976172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:06.976172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:06.976172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:06.976172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:06.976172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:06.976172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:06.976172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:06.976172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:06.976172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:06.976172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:06.976172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:21:06.976172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:06.976172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 515/1027] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Jun 7 08:21:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:06.982407 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24994e3-a4e4-4e6d-b063-faa70337d67b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:06.995267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6304233-b408-412c-afbb-19480fb2c6ee None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:07.032246 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441ab61-fe8b-48e5-8f43-2acd6ba4eacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:07.033024 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441ab61-fe8b-48e5-8f43-2acd6ba4eacb None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:07.033316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441ab61-fe8b-48e5-8f43-2acd6ba4eacb None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:07.033592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441ab61-fe8b-48e5-8f43-2acd6ba4eacb None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:07.034471 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441ab61-fe8b-48e5-8f43-2acd6ba4eacb None admin] RBAC: Adding query filter params (type=image) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:07.034592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441ab61-fe8b-48e5-8f43-2acd6ba4eacb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:07.035962 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441ab61-fe8b-48e5-8f43-2acd6ba4eacb None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:07.040852 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 513/1028] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:07.047167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441ab61-fe8b-48e5-8f43-2acd6ba4eacb None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:07.072408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:07.097944 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24994e3-a4e4-4e6d-b063-faa70337d67b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:07.112055 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:07.135181 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7af66f-a91b-4693-ac52-f339ff5ffb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:07.135875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7af66f-a91b-4693-ac52-f339ff5ffb80 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:07.136262 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7af66f-a91b-4693-ac52-f339ff5ffb80 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:07.136439 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7af66f-a91b-4693-ac52-f339ff5ffb80 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:07.137227 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7af66f-a91b-4693-ac52-f339ff5ffb80 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:07.137727 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7af66f-a91b-4693-ac52-f339ff5ffb80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:07.138933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7af66f-a91b-4693-ac52-f339ff5ffb80 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:07.147378 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 516/1029] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Jun 7 08:21:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:07.152848 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7af66f-a91b-4693-ac52-f339ff5ffb80 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:07.153256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441ab61-fe8b-48e5-8f43-2acd6ba4eacb None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:07.189319 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b627c46-82b3-463d-a603-761e63773abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:07.190054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b627c46-82b3-463d-a603-761e63773abb None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:07.190162 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b627c46-82b3-463d-a603-761e63773abb None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:07.190388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b627c46-82b3-463d-a603-761e63773abb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:07.191562 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b627c46-82b3-463d-a603-761e63773abb None admin] RBAC: Adding query filter params (interface=public, service_id=bd400bb961c24f6fb27b75222d7deba0, region_id=RegionOne) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:07.191880 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b627c46-82b3-463d-a603-761e63773abb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:07.193314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b627c46-82b3-463d-a603-761e63773abb None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:07.203191 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 514/1030] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Jun 7 08:21:07 2022] GET /identity/v3/endpoints?service_id=bd400bb961c24f6fb27b75222d7deba0&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:07.204102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b627c46-82b3-463d-a603-761e63773abb None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:07.263383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7af66f-a91b-4693-ac52-f339ff5ffb80 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:07.296096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:07.298955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de2d66-5c31-4ba8-a20d-5baf861cd427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:07.299810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de2d66-5c31-4ba8-a20d-5baf861cd427 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:07.300093 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de2d66-5c31-4ba8-a20d-5baf861cd427 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:07.300366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de2d66-5c31-4ba8-a20d-5baf861cd427 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:07.312035 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de2d66-5c31-4ba8-a20d-5baf861cd427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:07.312650 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b627c46-82b3-463d-a603-761e63773abb None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:07.313621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de2d66-5c31-4ba8-a20d-5baf861cd427 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:07.331995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-47de2d66-5c31-4ba8-a20d-5baf861cd427 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:07.331995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:07.331995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:07.331995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:07.331995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:07.331995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:07.331995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:07.331995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:07.331995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:07.331995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:07.331995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:07.331995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:07.331995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:21:07.331995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:21:07.331995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:21:07.332904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:21:07.332904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:07.332904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:07.332904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:07.332904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:07.332904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:07.332904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:07.332904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:07.332904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:07.332904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:07.332904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:07.332904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:07.332904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:07.332904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:07.332904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:07.332904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:07.332904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:07.333857 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:21:07.333857 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:21:07.333857 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:07.333857 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:07.333857 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 517/1031] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:07.338886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de2d66-5c31-4ba8-a20d-5baf861cd427 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:07.349305 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e544dec-8b49-4b20-a876-89750e6d72a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:07.349963 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e544dec-8b49-4b20-a876-89750e6d72a9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:07.350247 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e544dec-8b49-4b20-a876-89750e6d72a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:07.350523 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e544dec-8b49-4b20-a876-89750e6d72a9 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:07.351203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e544dec-8b49-4b20-a876-89750e6d72a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:07.352568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e544dec-8b49-4b20-a876-89750e6d72a9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:07.359610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 515/1032] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Jun 7 08:21:07 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:21:07.370292 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:07.467456 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de2d66-5c31-4ba8-a20d-5baf861cd427 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:07.504237 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456c8ba-08b8-4d28-adb5-75faffc356f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:07.504923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456c8ba-08b8-4d28-adb5-75faffc356f7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:07.505241 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456c8ba-08b8-4d28-adb5-75faffc356f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:07.505522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456c8ba-08b8-4d28-adb5-75faffc356f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:07.506262 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d456c8ba-08b8-4d28-adb5-75faffc356f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:07.506572 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d456c8ba-08b8-4d28-adb5-75faffc356f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:07.507973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d456c8ba-08b8-4d28-adb5-75faffc356f7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:07.521044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 518/1033] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:07.526378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e544dec-8b49-4b20-a876-89750e6d72a9 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:07.631800 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e544dec-8b49-4b20-a876-89750e6d72a9 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:07.665483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7caac5-e4f7-41ad-b6b0-234fbf71435a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:07.666275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7caac5-e4f7-41ad-b6b0-234fbf71435a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:07.666562 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7caac5-e4f7-41ad-b6b0-234fbf71435a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:07.666840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7caac5-e4f7-41ad-b6b0-234fbf71435a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:07.674679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7caac5-e4f7-41ad-b6b0-234fbf71435a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:07.676131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7caac5-e4f7-41ad-b6b0-234fbf71435a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:07.690691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-5e7caac5-e4f7-41ad-b6b0-234fbf71435a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:21:07.690691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:07.690691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:07.690691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:07.690691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:07.690691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:07.690691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:07.690691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:07.690691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:07.690691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:07.690691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:07.690691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:07.690691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:21:07.690691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:21:07.690691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:21:07.690691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:21:07.691642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:07.691642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:07.691642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:21:07.691642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:21:07.691642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:21:07.691642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:21:07.691642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:07.691642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:07.691642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:07.691642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:07.691642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:07.691642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:07.691642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:07.691642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:07.691642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:07.691642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:07.691642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:07.691642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:07.692940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:07.692940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:07.692940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:21:07.692940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:21:07.692940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:21:07.692940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:21:07.692940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:21:07.692940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:21:07.692940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:21:07.692940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:07.692940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 516/1034] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Jun 7 08:21:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:07.697486 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456c8ba-08b8-4d28-adb5-75faffc356f7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:07.788441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:07.803288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456c8ba-08b8-4d28-adb5-75faffc356f7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:07.843949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11a75d-b043-4da6-b69e-648969bd73e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:07.844626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11a75d-b043-4da6-b69e-648969bd73e9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:07.844907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11a75d-b043-4da6-b69e-648969bd73e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:07.845205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11a75d-b043-4da6-b69e-648969bd73e9 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:07.846039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11a75d-b043-4da6-b69e-648969bd73e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:07.846344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11a75d-b043-4da6-b69e-648969bd73e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:07.847753 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11a75d-b043-4da6-b69e-648969bd73e9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:07.869399 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 519/1035] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Jun 7 08:21:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:07.870670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7caac5-e4f7-41ad-b6b0-234fbf71435a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:07.884363 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:07.973988 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7caac5-e4f7-41ad-b6b0-234fbf71435a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:08.007366 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f5f7a-6e70-4d69-84fc-8fd38c7e0083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:08.008003 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f5f7a-6e70-4d69-84fc-8fd38c7e0083 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:08.008279 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f5f7a-6e70-4d69-84fc-8fd38c7e0083 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:08.008546 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f5f7a-6e70-4d69-84fc-8fd38c7e0083 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:08.075669 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f5f7a-6e70-4d69-84fc-8fd38c7e0083 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:08.077054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f5f7a-6e70-4d69-84fc-8fd38c7e0083 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:08.088081 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:08.176614 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 517/1036] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:07 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:21:08.181901 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11a75d-b043-4da6-b69e-648969bd73e9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:08.286118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11a75d-b043-4da6-b69e-648969bd73e9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:08.319682 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5dc9fd-66a2-46b8-af82-6492dcad0a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:08.320474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dc9fd-66a2-46b8-af82-6492dcad0a37 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:08.320755 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dc9fd-66a2-46b8-af82-6492dcad0a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:08.321052 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dc9fd-66a2-46b8-af82-6492dcad0a37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:08.331572 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5dc9fd-66a2-46b8-af82-6492dcad0a37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:08.332944 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5dc9fd-66a2-46b8-af82-6492dcad0a37 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:08.335423 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:08.350732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-cb5dc9fd-66a2-46b8-af82-6492dcad0a37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:08.350732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:08.350732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:08.350732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:08.350732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:08.350732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:08.350732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:08.350732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:08.350732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:08.350732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:08.350732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:08.350732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:08.350732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:21:08.350732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:21:08.350732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:21:08.351661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:21:08.351661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:08.351661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:08.351661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:08.351661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:08.351661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:08.351661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:08.351661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:08.351661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:08.351661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:08.351661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:08.351661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:08.351661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:08.351661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:08.351661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:08.351661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:08.351661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:08.352574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:21:08.352574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:21:08.352574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:08.352574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:08.352574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 520/1037] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:08.358215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f5f7a-6e70-4d69-84fc-8fd38c7e0083 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:08.455684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f5f7a-6e70-4d69-84fc-8fd38c7e0083 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:08.466525 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:08.490279 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e563ae7-af2d-48d0-ae67-c2296c7e9228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:08.491186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e563ae7-af2d-48d0-ae67-c2296c7e9228 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:08.491612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e563ae7-af2d-48d0-ae67-c2296c7e9228 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:08.492085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e563ae7-af2d-48d0-ae67-c2296c7e9228 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:08.492975 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e563ae7-af2d-48d0-ae67-c2296c7e9228 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:08.493516 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e563ae7-af2d-48d0-ae67-c2296c7e9228 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:08.495030 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e563ae7-af2d-48d0-ae67-c2296c7e9228 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:08.512001 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 518/1038] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:08.513755 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5dc9fd-66a2-46b8-af82-6492dcad0a37 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:08.615631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5dc9fd-66a2-46b8-af82-6492dcad0a37 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:08.654625 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d23788-333e-4ee4-9c72-8cd2b4c300b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:08.655766 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d23788-333e-4ee4-9c72-8cd2b4c300b1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:08.656202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d23788-333e-4ee4-9c72-8cd2b4c300b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:08.656659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d23788-333e-4ee4-9c72-8cd2b4c300b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:08.663553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d23788-333e-4ee4-9c72-8cd2b4c300b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:08.665166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d23788-333e-4ee4-9c72-8cd2b4c300b1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:08.679601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-05d23788-333e-4ee4-9c72-8cd2b4c300b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:21:08.679601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:08.679601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:08.679601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:08.679601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:08.679601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:08.679601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:08.679601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:08.679601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:08.679601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:08.679601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:08.679601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:08.679601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:21:08.679601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:21:08.679601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:21:08.680554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:21:08.680554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:08.680554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:08.680554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:08.680554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:08.680554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:08.680554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:08.680554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:08.680554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:08.680554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:08.680554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:08.680554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:08.680554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:08.680554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:08.680554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:08.680554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:08.680554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:08.681500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:21:08.681500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:21:08.681500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:21:08.681500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:21:08.681500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:21:08.681500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:21:08.681500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:21:08.681500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:08.681500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 521/1039] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:21:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:08.688183 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e563ae7-af2d-48d0-ae67-c2296c7e9228 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:08.786050 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e563ae7-af2d-48d0-ae67-c2296c7e9228 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:08.819312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184cd6a-c34f-4178-82e3-ed69301706a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:08.820222 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184cd6a-c34f-4178-82e3-ed69301706a1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:08.820705 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184cd6a-c34f-4178-82e3-ed69301706a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:08.821195 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184cd6a-c34f-4178-82e3-ed69301706a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:08.822212 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184cd6a-c34f-4178-82e3-ed69301706a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:08.822706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184cd6a-c34f-4178-82e3-ed69301706a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:08.824896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184cd6a-c34f-4178-82e3-ed69301706a1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:08.841538 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 519/1040] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:21:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:08.843901 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d23788-333e-4ee4-9c72-8cd2b4c300b1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:08.900850 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:08.945546 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d23788-333e-4ee4-9c72-8cd2b4c300b1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:08.950994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a291e-4247-4c42-a1fa-4ed6981acde4 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:08.951280 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a291e-4247-4c42-a1fa-4ed6981acde4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:08.951556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a291e-4247-4c42-a1fa-4ed6981acde4 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:08.952904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 520/1041] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:08.958387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e13ad-a781-45b6-a612-2226e23f0357 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:08.958730 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e13ad-a781-45b6-a612-2226e23f0357 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:08.959071 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e13ad-a781-45b6-a612-2226e23f0357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:08.976986 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-6c4e13ad-a781-45b6-a612-2226e23f0357 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:08.978182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6c4e13ad-a781-45b6-a612-2226e23f0357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:08.986138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb14b23-2ba5-4d11-8c78-fc84796f9953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:08.987012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb14b23-2ba5-4d11-8c78-fc84796f9953 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:08.987570 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb14b23-2ba5-4d11-8c78-fc84796f9953 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:08.988020 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb14b23-2ba5-4d11-8c78-fc84796f9953 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:08.989037 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb14b23-2ba5-4d11-8c78-fc84796f9953 None admin] RBAC: Adding query filter params (role.id=c782c67dcd6a46cc8c5b26a87d6d2dc3, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=ee42568b059b49a198ed2c01d5efdafd) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:08.989603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb14b23-2ba5-4d11-8c78-fc84796f9953 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:08.991183 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb14b23-2ba5-4d11-8c78-fc84796f9953 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:08.997505 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 522/1042] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:08 2022] GET /identity/v3/role_assignments?user.id=ee42568b059b49a198ed2c01d5efdafd&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=c782c67dcd6a46cc8c5b26a87d6d2dc3 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:09.013406 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:09.062745 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 521/1043] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:21:09.087578 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb14b23-2ba5-4d11-8c78-fc84796f9953 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:09.198468 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb14b23-2ba5-4d11-8c78-fc84796f9953 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:09.220967 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:09.234741 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1bf0b-31d1-4866-81ab-2119fb0151b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:09.235532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1bf0b-31d1-4866-81ab-2119fb0151b5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:09.235809 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1bf0b-31d1-4866-81ab-2119fb0151b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:09.236079 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1bf0b-31d1-4866-81ab-2119fb0151b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:09.237295 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1bf0b-31d1-4866-81ab-2119fb0151b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:09.238714 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1bf0b-31d1-4866-81ab-2119fb0151b5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:09.240554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 523/1044] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:09.249041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e13ad-a781-45b6-a612-2226e23f0357 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:09.351395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e13ad-a781-45b6-a612-2226e23f0357 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:09.385641 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8796521-4864-4116-a594-b6ce3b612713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:09.386433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8796521-4864-4116-a594-b6ce3b612713 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:09.386620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8796521-4864-4116-a594-b6ce3b612713 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:09.386893 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8796521-4864-4116-a594-b6ce3b612713 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:09.388139 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8796521-4864-4116-a594-b6ce3b612713 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:09.389628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8796521-4864-4116-a594-b6ce3b612713 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:09.481846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 522/1045] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jun 7 08:21:09 2022] POST /identity/v3/users => generated 285 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:21:09.665771 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2111bff-2c36-4e93-be0f-2db3e484635f None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:09.666279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2111bff-2c36-4e93-be0f-2db3e484635f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:09.666369 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2111bff-2c36-4e93-be0f-2db3e484635f None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:09.667833 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 524/1046] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:09.674342 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641523c-775e-4524-96a0-5ea459d89910 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:09.674720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641523c-775e-4524-96a0-5ea459d89910 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:09.675179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641523c-775e-4524-96a0-5ea459d89910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:09.697303 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-f641523c-775e-4524-96a0-5ea459d89910 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:09.698517 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f641523c-775e-4524-96a0-5ea459d89910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:09.710603 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:09.768223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 523/1047] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:09.790914 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2111bff-2c36-4e93-be0f-2db3e484635f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:09.894156 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2111bff-2c36-4e93-be0f-2db3e484635f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:09.927828 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35024ebc-700e-49cf-a177-e3e0c6046f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:09.928612 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35024ebc-700e-49cf-a177-e3e0c6046f28 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:09.928890 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35024ebc-700e-49cf-a177-e3e0c6046f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:09.929185 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35024ebc-700e-49cf-a177-e3e0c6046f28 None admin] PATH_INFO: `/v3/services/image` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:09.929824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35024ebc-700e-49cf-a177-e3e0c6046f28 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:09.931172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35024ebc-700e-49cf-a177-e3e0c6046f28 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:09.943074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-35024ebc-700e-49cf-a177-e3e0c6046f28 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 08:21:09.943074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:09.943074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:21:09.943074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:21:09.943074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:21:09.943074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:21:09.943074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:21:09.943074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:21:09.943074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 08:21:09.943074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:09.943074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:21:09.943074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:09.943074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:09.943074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:09.943074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:09.943074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:09.943074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:09.944110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:09.944110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:09.944110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:09.944110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:09.944110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:09.944110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:09.944110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:21:09.944110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:21:09.944110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:21:09.944110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:21:09.944110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:09.944110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:09.944110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:09.944110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:09.944110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:09.944110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:09.944985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:09.944985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:09.944985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:09.944985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:09.944985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:09.944985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:09.944985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:09.944985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:09.944985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:09.944985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:09.944985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:21:09.944985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:21:09.944985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 08:21:09.944985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:09.944985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 525/1048] 10.222.0.17 () {56 vars in 1105 bytes} [Tue Jun 7 08:21:09 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:09.952209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641523c-775e-4524-96a0-5ea459d89910 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:10.053034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641523c-775e-4524-96a0-5ea459d89910 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:10.088534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfde8ea-e0f5-4d9c-af20-f2e9b877cce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:10.089238 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfde8ea-e0f5-4d9c-af20-f2e9b877cce6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:10.089521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfde8ea-e0f5-4d9c-af20-f2e9b877cce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:10.089797 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfde8ea-e0f5-4d9c-af20-f2e9b877cce6 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:10.090573 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfde8ea-e0f5-4d9c-af20-f2e9b877cce6 None admin] RBAC: Adding query filter params (name=image) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:10.090879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfde8ea-e0f5-4d9c-af20-f2e9b877cce6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:10.092253 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfde8ea-e0f5-4d9c-af20-f2e9b877cce6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:10.101958 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 524/1049] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:10.103190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35024ebc-700e-49cf-a177-e3e0c6046f28 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:10.159062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:10.211603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35024ebc-700e-49cf-a177-e3e0c6046f28 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:10.246965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349e997-763e-466b-9431-61f9f188a84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:10.247637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349e997-763e-466b-9431-61f9f188a84a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:10.247922 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349e997-763e-466b-9431-61f9f188a84a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:10.248198 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349e997-763e-466b-9431-61f9f188a84a None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:10.248904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8349e997-763e-466b-9431-61f9f188a84a None admin] RBAC: Adding query filter params (type=image) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:10.249244 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8349e997-763e-466b-9431-61f9f188a84a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:10.250612 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8349e997-763e-466b-9431-61f9f188a84a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:10.256151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 526/1050] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:10.261556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfde8ea-e0f5-4d9c-af20-f2e9b877cce6 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:10.274119 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:10.362992 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfde8ea-e0f5-4d9c-af20-f2e9b877cce6 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:10.396520 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974be40-3bdf-43ed-87a5-d10112ab6903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:10.397256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974be40-3bdf-43ed-87a5-d10112ab6903 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:10.397493 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974be40-3bdf-43ed-87a5-d10112ab6903 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:10.397771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974be40-3bdf-43ed-87a5-d10112ab6903 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:10.398524 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974be40-3bdf-43ed-87a5-d10112ab6903 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:10.399892 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974be40-3bdf-43ed-87a5-d10112ab6903 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:10.498248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 525/1051] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jun 7 08:21:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:21:11.282577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5462333-1e58-406a-aacf-04d1af6bb04e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.283133 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5462333-1e58-406a-aacf-04d1af6bb04e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.283133 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5462333-1e58-406a-aacf-04d1af6bb04e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.284643 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 527/1052] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:11.289721 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a724582-8ae6-4f5b-8966-63bc89325fe0 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.290013 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a724582-8ae6-4f5b-8966-63bc89325fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.290291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a724582-8ae6-4f5b-8966-63bc89325fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.307370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-6a724582-8ae6-4f5b-8966-63bc89325fe0 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:11.308564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6a724582-8ae6-4f5b-8966-63bc89325fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:11.370399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:11.385041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 526/1053] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:11.407386 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5462333-1e58-406a-aacf-04d1af6bb04e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:11.516297 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5462333-1e58-406a-aacf-04d1af6bb04e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:11.533529 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:11.551196 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707605e2-2f64-4fde-9c39-188c39e9d4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:11.551998 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707605e2-2f64-4fde-9c39-188c39e9d4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.552278 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707605e2-2f64-4fde-9c39-188c39e9d4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.552548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707605e2-2f64-4fde-9c39-188c39e9d4b3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.563064 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707605e2-2f64-4fde-9c39-188c39e9d4b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:11.571646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-707605e2-2f64-4fde-9c39-188c39e9d4b3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:21:11.571646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:11.571646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:11.571646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:11.571646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:11.571646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:11.571646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:11.571646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:11.571646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:11.571646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:11.571646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:11.571646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:11.571646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:11.571646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:11.571646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:11.571646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:11.572787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:11.572787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:11.572787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:11.572787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:11.572787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:11.572787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:11.572787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:11.572787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:11.572787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:11.572787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:11.572787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:11.572787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:11.572787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:11.572787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:11.572787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:11.572787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:11.573867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:11.573867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:11.573867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:11.573867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:11.573867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:11.573867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:11.573867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:11.573867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:11.573867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:11.573867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:11.573867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:21:11.573867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:11.577046 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 528/1054] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Jun 7 08:21:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:11.582723 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a724582-8ae6-4f5b-8966-63bc89325fe0 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:11.686035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a724582-8ae6-4f5b-8966-63bc89325fe0 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:11.721046 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ce7b8-812c-45e3-99b6-e4eff37dd0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:11.721721 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ce7b8-812c-45e3-99b6-e4eff37dd0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.722005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ce7b8-812c-45e3-99b6-e4eff37dd0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.722283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ce7b8-812c-45e3-99b6-e4eff37dd0d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.723029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ce7b8-812c-45e3-99b6-e4eff37dd0d7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:11.723402 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ce7b8-812c-45e3-99b6-e4eff37dd0d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:11.724887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ce7b8-812c-45e3-99b6-e4eff37dd0d7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:11.737767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 527/1055] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Jun 7 08:21:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:11.739188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707605e2-2f64-4fde-9c39-188c39e9d4b3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:11.805735 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9df489-a4b8-480f-abe5-606fbeb70573 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.806032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9df489-a4b8-480f-abe5-606fbeb70573 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.806220 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9df489-a4b8-480f-abe5-606fbeb70573 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.807697 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 528/1056] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:21:11.813042 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f6b39-ea76-4324-a089-b81809fa8d04 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.813331 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f6b39-ea76-4324-a089-b81809fa8d04 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.813611 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f6b39-ea76-4324-a089-b81809fa8d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.832315 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ab1f6b39-ea76-4324-a089-b81809fa8d04 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:11.833543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ab1f6b39-ea76-4324-a089-b81809fa8d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:11.842526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707605e2-2f64-4fde-9c39-188c39e9d4b3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:11.876746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc16bf-7ac4-4e0d-a7c4-f0dba6274961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:11.877562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc16bf-7ac4-4e0d-a7c4-f0dba6274961 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:11.878088 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc16bf-7ac4-4e0d-a7c4-f0dba6274961 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:11.878088 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc16bf-7ac4-4e0d-a7c4-f0dba6274961 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:11.885604 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:11.890242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc16bf-7ac4-4e0d-a7c4-f0dba6274961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:11.891733 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc16bf-7ac4-4e0d-a7c4-f0dba6274961 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:11.903261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 529/1057] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:11.908743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-bbcc16bf-7ac4-4e0d-a7c4-f0dba6274961 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:11.908743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:11.908743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:11.908743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:11.908743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:11.908743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:11.908743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:11.908743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:11.908743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:11.908743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:11.908743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:11.908743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:11.908743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:21:11.908743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:21:11.908743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:21:11.909806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:21:11.909806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:11.909806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:11.909806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:11.909806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:11.909806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:11.909806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:11.909806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:11.909806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:11.909806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:11.909806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:11.909806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:11.909806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:11.909806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:11.909806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:11.909806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:11.909806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:11.910735 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:21:11.910735 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:21:11.910735 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:11.910735 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:11.910735 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 529/1058] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:11.915387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f6b39-ea76-4324-a089-b81809fa8d04 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:11.928597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc16bf-7ac4-4e0d-a7c4-f0dba6274961 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:11.959334 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:12.021306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f6b39-ea76-4324-a089-b81809fa8d04 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.033051 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc16bf-7ac4-4e0d-a7c4-f0dba6274961 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.057588 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0fea0-7215-4bdf-a9d4-bb0fc8ee2b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.058270 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0fea0-7215-4bdf-a9d4-bb0fc8ee2b76 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.058553 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0fea0-7215-4bdf-a9d4-bb0fc8ee2b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.058823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0fea0-7215-4bdf-a9d4-bb0fc8ee2b76 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.060069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0fea0-7215-4bdf-a9d4-bb0fc8ee2b76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:12.060374 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0fea0-7215-4bdf-a9d4-bb0fc8ee2b76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:12.061768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0fea0-7215-4bdf-a9d4-bb0fc8ee2b76 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:12.069999 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd72610-e1ca-485a-8e61-cc4514f4e6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.071007 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd72610-e1ca-485a-8e61-cc4514f4e6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.071471 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd72610-e1ca-485a-8e61-cc4514f4e6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.071944 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd72610-e1ca-485a-8e61-cc4514f4e6f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.072989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 530/1059] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:12.078732 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0fea0-7215-4bdf-a9d4-bb0fc8ee2b76 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:12.088169 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd72610-e1ca-485a-8e61-cc4514f4e6f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:12.096908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-3fd72610-e1ca-485a-8e61-cc4514f4e6f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:12.096908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:12.096908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:12.096908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:12.096908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:12.096908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:12.096908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:12.096908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:12.096908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:12.096908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:12.096908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:12.096908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:12.096908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:12.096908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:12.096908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:12.096908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:12.097996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:12.097996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:12.097996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:12.097996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:12.097996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:12.097996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:12.097996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:12.097996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:12.097996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:12.097996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:12.097996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:12.097996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:12.097996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:12.097996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:12.097996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:12.097996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:12.098832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:12.098832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:12.098832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:12.098832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:12.098832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:12.098832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:12.098832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:12.098832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:12.098832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:12.098832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:12.098832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:12.098832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:12.098832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 530/1060] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:21:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:12.103906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd72610-e1ca-485a-8e61-cc4514f4e6f5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:12.130177 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:12.193749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0fea0-7215-4bdf-a9d4-bb0fc8ee2b76 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.194228 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:12.206083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd72610-e1ca-485a-8e61-cc4514f4e6f5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.228417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b4de4-0c06-4140-8be3-54da8b466e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.229236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b4de4-0c06-4140-8be3-54da8b466e5f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.229518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b4de4-0c06-4140-8be3-54da8b466e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.229791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b4de4-0c06-4140-8be3-54da8b466e5f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.238162 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b4de4-0c06-4140-8be3-54da8b466e5f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:12.239638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b4de4-0c06-4140-8be3-54da8b466e5f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:12.248818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e02065-409d-4c97-9364-6a46cedb2676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.249519 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e02065-409d-4c97-9364-6a46cedb2676 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.249804 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e02065-409d-4c97-9364-6a46cedb2676 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.250081 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e02065-409d-4c97-9364-6a46cedb2676 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.250823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e02065-409d-4c97-9364-6a46cedb2676 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:12.251132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e02065-409d-4c97-9364-6a46cedb2676 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:12.252486 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e02065-409d-4c97-9364-6a46cedb2676 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:12.254203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-c62b4de4-0c06-4140-8be3-54da8b466e5f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:21:12.254203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:12.254203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:12.254203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:12.254203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:12.254203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:12.254203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:12.254203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:12.254203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:12.254203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:12.254203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:12.254203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:12.254203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:21:12.254203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:21:12.254203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:21:12.254203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:21:12.255178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:12.255178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:12.255178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:21:12.255178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:21:12.255178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:21:12.255178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:21:12.255178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:12.255178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:12.255178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:12.255178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:12.255178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:12.255178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:12.255178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:12.255178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:12.255178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:12.255178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:12.255178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:12.255178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:12.256553 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:12.256553 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:12.256553 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:21:12.256553 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:21:12.256553 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:21:12.256553 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:21:12.256553 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:21:12.256553 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:21:12.256553 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:21:12.256553 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:12.256553 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 531/1061] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Jun 7 08:21:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:12.261227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b4de4-0c06-4140-8be3-54da8b466e5f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:12.264553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 531/1062] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:12.270201 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e02065-409d-4c97-9364-6a46cedb2676 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:12.364819 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b4de4-0c06-4140-8be3-54da8b466e5f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.367897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:12.378034 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e02065-409d-4c97-9364-6a46cedb2676 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.400448 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0551d94-a5df-41e9-ae3d-288ffcf108fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.401256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0551d94-a5df-41e9-ae3d-288ffcf108fc None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.401540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0551d94-a5df-41e9-ae3d-288ffcf108fc None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.401818 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0551d94-a5df-41e9-ae3d-288ffcf108fc None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.402671 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0551d94-a5df-41e9-ae3d-288ffcf108fc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:12.402988 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0551d94-a5df-41e9-ae3d-288ffcf108fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:12.404424 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0551d94-a5df-41e9-ae3d-288ffcf108fc None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:12.411530 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73d96f-0e46-4f65-b01f-f404066e2439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.412409 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73d96f-0e46-4f65-b01f-f404066e2439 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.412695 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73d96f-0e46-4f65-b01f-f404066e2439 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.412971 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73d96f-0e46-4f65-b01f-f404066e2439 None admin] PATH_INFO: `/v3/users/ac218165cb9a4b47a216d9f53592c242` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.425459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 532/1063] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Jun 7 08:21:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:12.429431 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73d96f-0e46-4f65-b01f-f404066e2439 None admin] RBAC: Authorizing `identity:get_user(user_id=ac218165cb9a4b47a216d9f53592c242)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:12.430935 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73d96f-0e46-4f65-b01f-f404066e2439 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:12.431046 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0551d94-a5df-41e9-ae3d-288ffcf108fc None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:12.438234 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 532/1064] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:12 2022] GET /identity/v3/users/ac218165cb9a4b47a216d9f53592c242 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:12.439260 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73d96f-0e46-4f65-b01f-f404066e2439 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:12.450471 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:12.543113 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73d96f-0e46-4f65-b01f-f404066e2439 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.546597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0551d94-a5df-41e9-ae3d-288ffcf108fc None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.580113 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0ac43-3c6e-46b6-8dd9-83193093eada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.580762 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0ac43-3c6e-46b6-8dd9-83193093eada None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.581086 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0ac43-3c6e-46b6-8dd9-83193093eada None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.581345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0ac43-3c6e-46b6-8dd9-83193093eada None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.582509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5c037-173c-4792-98a2-bdd3c826def3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.583505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5c037-173c-4792-98a2-bdd3c826def3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.584406 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5c037-173c-4792-98a2-bdd3c826def3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.584897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5c037-173c-4792-98a2-bdd3c826def3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.596113 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb5c037-173c-4792-98a2-bdd3c826def3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:12.597614 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb5c037-173c-4792-98a2-bdd3c826def3 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:12.615555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-1cb5c037-173c-4792-98a2-bdd3c826def3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:12.615555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:12.615555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:12.615555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:12.615555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:12.615555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:12.615555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:12.615555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:12.615555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:12.615555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:12.615555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:12.615555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:12.615555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:21:12.615555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:21:12.615555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:21:12.616482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:21:12.616482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:12.616482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:12.616482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:12.616482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:12.616482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:12.616482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:12.616482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:12.616482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:12.616482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:12.616482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:12.616482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:12.616482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:12.616482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:12.616482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:12.616482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:12.616482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:12.617519 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:21:12.617519 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:21:12.617519 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:12.617519 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:12.617519 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 533/1065] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:12.622377 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5c037-173c-4792-98a2-bdd3c826def3 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:12.639206 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:12.653667 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0ac43-3c6e-46b6-8dd9-83193093eada None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:12.655151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0ac43-3c6e-46b6-8dd9-83193093eada None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:12.664903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:12.726704 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5c037-173c-4792-98a2-bdd3c826def3 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.759775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 533/1066] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 08:21:12.764584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a642c-3c6b-47e3-a4fc-05a73aa3897b None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.764871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a642c-3c6b-47e3-a4fc-05a73aa3897b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.765178 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a642c-3c6b-47e3-a4fc-05a73aa3897b None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.766468 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 534/1067] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:21:12.767160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a642c-3c6b-47e3-a4fc-05a73aa3897b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:12.768235 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83694f-b48e-4ade-892c-06c10b6e0da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.768895 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83694f-b48e-4ade-892c-06c10b6e0da5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.769204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83694f-b48e-4ade-892c-06c10b6e0da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.769478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83694f-b48e-4ade-892c-06c10b6e0da5 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.770392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83694f-b48e-4ade-892c-06c10b6e0da5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:12.771536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83694f-b48e-4ade-892c-06c10b6e0da5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:12.772903 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83694f-b48e-4ade-892c-06c10b6e0da5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:12.784811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 534/1068] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:12.786069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f878a-0c83-4f7d-9fe2-d7dc81cbd0e7 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.786359 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f878a-0c83-4f7d-9fe2-d7dc81cbd0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.786639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f878a-0c83-4f7d-9fe2-d7dc81cbd0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.803140 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-587f878a-0c83-4f7d-9fe2-d7dc81cbd0e7 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:12.804315 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-587f878a-0c83-4f7d-9fe2-d7dc81cbd0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:12.868139 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a642c-3c6b-47e3-a4fc-05a73aa3897b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:12.869691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 535/1069] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:12.870504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f878a-0c83-4f7d-9fe2-d7dc81cbd0e7 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:12.874201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:12.877767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:12.902601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789705c-1836-4a40-89c8-9705f20e9952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:12.903999 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789705c-1836-4a40-89c8-9705f20e9952 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:12.904311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789705c-1836-4a40-89c8-9705f20e9952 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:12.904581 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789705c-1836-4a40-89c8-9705f20e9952 None admin] PATH_INFO: `/v3/projects/6a50c552a3034a138ae711d82c59f9f9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:12.905765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1789705c-1836-4a40-89c8-9705f20e9952 None admin] RBAC: Authorizing `identity:get_project(project_id=6a50c552a3034a138ae711d82c59f9f9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:12.907188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1789705c-1836-4a40-89c8-9705f20e9952 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:12.908884 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 535/1070] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:12 2022] GET /identity/v3/projects/6a50c552a3034a138ae711d82c59f9f9 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:12.909672 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789705c-1836-4a40-89c8-9705f20e9952 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:12.969124 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f878a-0c83-4f7d-9fe2-d7dc81cbd0e7 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:13.001537 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f9728-d873-4b9f-84ae-6240b26be75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:13.002535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f9728-d873-4b9f-84ae-6240b26be75a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:13.002814 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f9728-d873-4b9f-84ae-6240b26be75a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:13.003090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f9728-d873-4b9f-84ae-6240b26be75a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:13.006551 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789705c-1836-4a40-89c8-9705f20e9952 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:13.009623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f9728-d873-4b9f-84ae-6240b26be75a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:13.011058 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f9728-d873-4b9f-84ae-6240b26be75a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:13.025159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-659f9728-d873-4b9f-84ae-6240b26be75a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:21:13.025159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:13.025159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:13.025159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:13.025159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:13.025159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:13.025159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:13.025159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:13.025159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:13.025159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:13.025159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:13.025159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:13.025159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:21:13.025159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:21:13.025159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:21:13.026084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:21:13.026084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:13.026084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:13.026084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:13.026084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:13.026084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:13.026084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:13.026084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:13.026084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:13.026084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:13.026084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:13.026084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:13.026084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:13.026084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:13.026084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:13.026084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:13.026084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:13.026969 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:21:13.026969 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:21:13.026969 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:21:13.026969 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:21:13.026969 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:21:13.026969 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:21:13.026969 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:21:13.026969 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:13.026969 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 536/1071] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:21:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:13.027450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f9728-d873-4b9f-84ae-6240b26be75a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:13.043989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6183d3-0bde-47b6-8921-7ba228ab8e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:13.044634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6183d3-0bde-47b6-8921-7ba228ab8e5a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:13.044917 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6183d3-0bde-47b6-8921-7ba228ab8e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:13.045229 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6183d3-0bde-47b6-8921-7ba228ab8e5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:13.088372 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:13.089344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:13.111477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6183d3-0bde-47b6-8921-7ba228ab8e5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:13.112937 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6183d3-0bde-47b6-8921-7ba228ab8e5a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:13.125873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f9728-d873-4b9f-84ae-6240b26be75a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:13.159165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2780dd5-cff9-4ed0-91e5-bce02d8a3ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:13.159804 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2780dd5-cff9-4ed0-91e5-bce02d8a3ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:13.160087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2780dd5-cff9-4ed0-91e5-bce02d8a3ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:13.160363 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2780dd5-cff9-4ed0-91e5-bce02d8a3ac8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:13.161222 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2780dd5-cff9-4ed0-91e5-bce02d8a3ac8 None admin] RBAC: Adding query filter params (role.id=8c63c00418a346b29f7db3f87a404e70, scope.project.id=6a50c552a3034a138ae711d82c59f9f9, user.id=ac218165cb9a4b47a216d9f53592c242) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:13.161534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2780dd5-cff9-4ed0-91e5-bce02d8a3ac8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:13.162951 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2780dd5-cff9-4ed0-91e5-bce02d8a3ac8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:13.168380 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 537/1072] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:13 2022] GET /identity/v3/role_assignments?user.id=ac218165cb9a4b47a216d9f53592c242&scope.project.id=6a50c552a3034a138ae711d82c59f9f9&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:13.169174 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2780dd5-cff9-4ed0-91e5-bce02d8a3ac8 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:13.227402 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 536/1073] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:21:13.233828 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6183d3-0bde-47b6-8921-7ba228ab8e5a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:13.287701 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2780dd5-cff9-4ed0-91e5-bce02d8a3ac8 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:13.315803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:13.321913 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:13.324206 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e877b5f-ef4f-4b01-b372-eb73da39efc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:13.325455 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e877b5f-ef4f-4b01-b372-eb73da39efc6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:13.325788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e877b5f-ef4f-4b01-b372-eb73da39efc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:13.326101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e877b5f-ef4f-4b01-b372-eb73da39efc6 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:13.326875 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e877b5f-ef4f-4b01-b372-eb73da39efc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:13.327186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e877b5f-ef4f-4b01-b372-eb73da39efc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:13.328659 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e877b5f-ef4f-4b01-b372-eb73da39efc6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:13.341621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 538/1074] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:21:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:13.347758 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e877b5f-ef4f-4b01-b372-eb73da39efc6 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:13.353713 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6183d3-0bde-47b6-8921-7ba228ab8e5a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:13.392552 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed69685-e170-4e1b-b5b8-e390af943c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:13.393336 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed69685-e170-4e1b-b5b8-e390af943c76 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:13.393599 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed69685-e170-4e1b-b5b8-e390af943c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:13.393873 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed69685-e170-4e1b-b5b8-e390af943c76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:13.404560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed69685-e170-4e1b-b5b8-e390af943c76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:13.406296 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed69685-e170-4e1b-b5b8-e390af943c76 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:13.425028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1ed69685-e170-4e1b-b5b8-e390af943c76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:13.425028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:13.425028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:13.425028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:13.425028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:13.425028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:13.425028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:13.425028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:13.425028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:13.425028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:13.425028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:13.425028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:13.425028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:21:13.425028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:21:13.425028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:21:13.425972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:21:13.425972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:13.425972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:13.425972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:13.425972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:13.425972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:13.425972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:13.425972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:13.425972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:13.425972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:13.425972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:13.425972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:13.425972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:13.425972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:13.425972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:13.425972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:13.425972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:13.427872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:21:13.427872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:21:13.427872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:13.427872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:13.427872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 537/1075] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:13.431940 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed69685-e170-4e1b-b5b8-e390af943c76 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:13.458265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e877b5f-ef4f-4b01-b372-eb73da39efc6 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:13.497432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5805ec0-0fd7-496d-89db-404dc8d0253d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:13.498320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5805ec0-0fd7-496d-89db-404dc8d0253d None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:13.498603 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5805ec0-0fd7-496d-89db-404dc8d0253d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:13.498878 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5805ec0-0fd7-496d-89db-404dc8d0253d None admin] PATH_INFO: `/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/ee42568b059b49a198ed2c01d5efdafd/roles/c782c67dcd6a46cc8c5b26a87d6d2dc3` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:13.501162 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5805ec0-0fd7-496d-89db-404dc8d0253d None admin] RBAC: Authorizing `identity:create_grant(project_id=afc8781e5f864d4a9374300ee9b5193b, user_id=ee42568b059b49a198ed2c01d5efdafd, role_id=c782c67dcd6a46cc8c5b26a87d6d2dc3)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:13.502600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5805ec0-0fd7-496d-89db-404dc8d0253d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:13.539971 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed69685-e170-4e1b-b5b8-e390af943c76 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:13.569044 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 539/1076] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:21:13 2022] PUT /identity/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/ee42568b059b49a198ed2c01d5efdafd/roles/c782c67dcd6a46cc8c5b26a87d6d2dc3 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:21:13.569298 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:13.577956 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac3e24-1724-4d0c-91d2-5b784bfe17db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:13.578647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac3e24-1724-4d0c-91d2-5b784bfe17db None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:13.578930 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac3e24-1724-4d0c-91d2-5b784bfe17db None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:13.579205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac3e24-1724-4d0c-91d2-5b784bfe17db None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:13.579946 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac3e24-1724-4d0c-91d2-5b784bfe17db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:13.580259 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac3e24-1724-4d0c-91d2-5b784bfe17db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:13.581832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac3e24-1724-4d0c-91d2-5b784bfe17db None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:13.601350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 538/1077] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:15.401423 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b3f15-6e02-4724-9763-8799959626f5 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:15.401946 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b3f15-6e02-4724-9763-8799959626f5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:15.402057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b3f15-6e02-4724-9763-8799959626f5 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:15.403332 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 540/1078] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:15.408804 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f2202-f0b0-4ac2-96be-3ec45f5d3c8f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:15.409301 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f2202-f0b0-4ac2-96be-3ec45f5d3c8f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:15.409394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f2202-f0b0-4ac2-96be-3ec45f5d3c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:15.426385 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6f4f2202-f0b0-4ac2-96be-3ec45f5d3c8f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:15.427556 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6f4f2202-f0b0-4ac2-96be-3ec45f5d3c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:15.494233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 539/1079] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:15.522266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b3f15-6e02-4724-9763-8799959626f5 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:15.573731 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:15.623806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b3f15-6e02-4724-9763-8799959626f5 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:15.656598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cc06d-d4bb-45ae-83ea-37f3ddafecfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:15.657416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cc06d-d4bb-45ae-83ea-37f3ddafecfc None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:15.657697 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cc06d-d4bb-45ae-83ea-37f3ddafecfc None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:15.657968 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cc06d-d4bb-45ae-83ea-37f3ddafecfc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:15.667608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cc06d-d4bb-45ae-83ea-37f3ddafecfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:15.676275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-b16cc06d-d4bb-45ae-83ea-37f3ddafecfc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:15.676275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:15.676275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:15.676275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:15.676275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:15.676275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:15.676275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:15.676275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:15.676275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:15.676275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:15.676275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:15.676275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:15.676275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:15.676275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:15.676275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:15.676275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:15.677314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:15.677314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:15.677314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:15.677314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:15.677314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:15.677314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:15.677314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:15.677314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:15.677314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:15.677314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:15.677314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:15.677314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:15.677314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:15.677314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:15.677314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:15.677314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:15.685767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:15.685767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:15.685767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:15.685767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:15.685767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:15.685767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:15.685767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:15.685767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:15.685767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:15.685767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:15.685767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:15.685767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:15.685767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 541/1080] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:21:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:15.686534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f2202-f0b0-4ac2-96be-3ec45f5d3c8f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:15.778079 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f2202-f0b0-4ac2-96be-3ec45f5d3c8f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:15.795002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:15.811294 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5999d7a-65d6-4d1a-85f3-a67d82e5ef3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:15.811936 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5999d7a-65d6-4d1a-85f3-a67d82e5ef3b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:15.812210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5999d7a-65d6-4d1a-85f3-a67d82e5ef3b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:15.812479 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5999d7a-65d6-4d1a-85f3-a67d82e5ef3b None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:15.813073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b70f6-bf8d-4beb-a100-546bf39046a7 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:15.813246 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5999d7a-65d6-4d1a-85f3-a67d82e5ef3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:15.813348 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b70f6-bf8d-4beb-a100-546bf39046a7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:15.813549 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5999d7a-65d6-4d1a-85f3-a67d82e5ef3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:15.813648 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b70f6-bf8d-4beb-a100-546bf39046a7 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:15.814903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5999d7a-65d6-4d1a-85f3-a67d82e5ef3b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:15.814991 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 542/1081] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:15.819742 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6774316-51a3-4cd4-8b0d-6e9011771718 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:15.820024 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6774316-51a3-4cd4-8b0d-6e9011771718 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:15.820310 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6774316-51a3-4cd4-8b0d-6e9011771718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:15.823230 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 540/1082] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:15.828712 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5999d7a-65d6-4d1a-85f3-a67d82e5ef3b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:15.837593 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e6774316-51a3-4cd4-8b0d-6e9011771718 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:15.838763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e6774316-51a3-4cd4-8b0d-6e9011771718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:15.904861 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 543/1083] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:21:15.926508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5999d7a-65d6-4d1a-85f3-a67d82e5ef3b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:15.928256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6774316-51a3-4cd4-8b0d-6e9011771718 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:15.959588 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be743a95-c9c6-4309-8f07-0296c69e8d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:15.960450 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be743a95-c9c6-4309-8f07-0296c69e8d86 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:15.960728 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be743a95-c9c6-4309-8f07-0296c69e8d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:15.961012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be743a95-c9c6-4309-8f07-0296c69e8d86 None admin] PATH_INFO: `/v3/users/ac218165cb9a4b47a216d9f53592c242` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:15.962213 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be743a95-c9c6-4309-8f07-0296c69e8d86 None admin] RBAC: Authorizing `identity:get_user(user_id=ac218165cb9a4b47a216d9f53592c242)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:15.963630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be743a95-c9c6-4309-8f07-0296c69e8d86 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:15.965872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 541/1084] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:15 2022] GET /identity/v3/users/ac218165cb9a4b47a216d9f53592c242 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:15.966860 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:15.971342 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be743a95-c9c6-4309-8f07-0296c69e8d86 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:16.019166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:16.025127 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6774316-51a3-4cd4-8b0d-6e9011771718 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:16.058390 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bca4ee-bfd1-49d4-a21c-0cb2b23fc216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:16.059118 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bca4ee-bfd1-49d4-a21c-0cb2b23fc216 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:16.059393 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bca4ee-bfd1-49d4-a21c-0cb2b23fc216 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:16.059665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bca4ee-bfd1-49d4-a21c-0cb2b23fc216 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:16.068770 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be743a95-c9c6-4309-8f07-0296c69e8d86 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:16.069345 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bca4ee-bfd1-49d4-a21c-0cb2b23fc216 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:16.077809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-47bca4ee-bfd1-49d4-a21c-0cb2b23fc216 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:21:16.077809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:16.077809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:16.077809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:16.077809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:16.077809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:16.077809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:16.077809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:16.077809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:16.077809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:16.077809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:16.077809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:16.077809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:16.077809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:16.077809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:16.077809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:16.078942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:16.078942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:16.078942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:16.078942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:16.078942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:16.078942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:16.078942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:16.078942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:16.078942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:16.078942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:16.078942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:16.078942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:16.078942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:16.078942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:16.078942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:16.078942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:16.084813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:16.084813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:16.084813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:16.084813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:16.084813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:16.084813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:16.084813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:16.084813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:16.084813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:16.084813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:16.084813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 08:21:16.084813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:16.084813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 544/1085] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Jun 7 08:21:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:16.084813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bca4ee-bfd1-49d4-a21c-0cb2b23fc216 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:16.101552 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4260c9-495f-4ae1-9ec2-43a8910297e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:16.102400 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4260c9-495f-4ae1-9ec2-43a8910297e2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:16.102680 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4260c9-495f-4ae1-9ec2-43a8910297e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:16.102948 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4260c9-495f-4ae1-9ec2-43a8910297e2 None admin] PATH_INFO: `/v3/projects/6a50c552a3034a138ae711d82c59f9f9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:16.104089 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4260c9-495f-4ae1-9ec2-43a8910297e2 None admin] RBAC: Authorizing `identity:get_project(project_id=6a50c552a3034a138ae711d82c59f9f9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:16.105559 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4260c9-495f-4ae1-9ec2-43a8910297e2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:16.107252 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 542/1086] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:15 2022] GET /identity/v3/projects/6a50c552a3034a138ae711d82c59f9f9 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:16.112978 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4260c9-495f-4ae1-9ec2-43a8910297e2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:16.180296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:16.184460 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bca4ee-bfd1-49d4-a21c-0cb2b23fc216 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:16.208153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4260c9-495f-4ae1-9ec2-43a8910297e2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:16.220970 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff7c31-cc64-4866-842b-c470918bbb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:16.221654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff7c31-cc64-4866-842b-c470918bbb7a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:16.221937 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff7c31-cc64-4866-842b-c470918bbb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:16.222209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff7c31-cc64-4866-842b-c470918bbb7a None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:16.222944 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff7c31-cc64-4866-842b-c470918bbb7a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:16.223245 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff7c31-cc64-4866-842b-c470918bbb7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:16.224184 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:16.224603 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff7c31-cc64-4866-842b-c470918bbb7a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:16.233167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 545/1087] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Jun 7 08:21:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:16.238454 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff7c31-cc64-4866-842b-c470918bbb7a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:16.241195 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd267ba4-bce2-4cb3-8aba-880b10ac9048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:16.242086 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd267ba4-bce2-4cb3-8aba-880b10ac9048 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:16.242366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd267ba4-bce2-4cb3-8aba-880b10ac9048 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:16.242640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd267ba4-bce2-4cb3-8aba-880b10ac9048 None admin] PATH_INFO: `/v3/projects/6a50c552a3034a138ae711d82c59f9f9/users/ac218165cb9a4b47a216d9f53592c242/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:16.244632 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd267ba4-bce2-4cb3-8aba-880b10ac9048 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a50c552a3034a138ae711d82c59f9f9, user_id=ac218165cb9a4b47a216d9f53592c242, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:16.246230 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd267ba4-bce2-4cb3-8aba-880b10ac9048 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:16.309955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 543/1088] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:21:16 2022] PUT /identity/v3/projects/6a50c552a3034a138ae711d82c59f9f9/users/ac218165cb9a4b47a216d9f53592c242/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:21:16.336949 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff7c31-cc64-4866-842b-c470918bbb7a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:16.374496 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a481057-a054-4411-8e5a-5697f10b0ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:16.375323 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a481057-a054-4411-8e5a-5697f10b0ead None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:16.375608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a481057-a054-4411-8e5a-5697f10b0ead None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:16.375882 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a481057-a054-4411-8e5a-5697f10b0ead None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:16.388757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a481057-a054-4411-8e5a-5697f10b0ead None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:16.390283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a481057-a054-4411-8e5a-5697f10b0ead None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:16.408965 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-8a481057-a054-4411-8e5a-5697f10b0ead None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:16.408965 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:16.408965 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:16.408965 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:16.408965 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:16.408965 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:16.408965 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:16.408965 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:16.408965 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:16.408965 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:16.408965 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:16.408965 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:16.408965 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:21:16.408965 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:21:16.408965 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:21:16.410571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:21:16.410571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:16.410571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:16.410571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:16.410571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:16.410571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:16.410571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:16.410571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:16.410571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:16.410571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:16.410571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:16.410571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:16.410571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:16.410571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:16.410571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:16.410571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:16.410571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:16.411479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:21:16.411479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:21:16.411479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:16.411479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:16.411479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 546/1089] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:16.416378 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd267ba4-bce2-4cb3-8aba-880b10ac9048 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:16.535770 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd267ba4-bce2-4cb3-8aba-880b10ac9048 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:16.569020 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214815f9-93e6-4929-81e2-dac67ac150ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:16.569609 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214815f9-93e6-4929-81e2-dac67ac150ed None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:16.569889 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214815f9-93e6-4929-81e2-dac67ac150ed None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:16.570161 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214815f9-93e6-4929-81e2-dac67ac150ed None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:16.570871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214815f9-93e6-4929-81e2-dac67ac150ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:16.571171 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214815f9-93e6-4929-81e2-dac67ac150ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:16.572531 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214815f9-93e6-4929-81e2-dac67ac150ed None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:16.584624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 544/1090] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:16.590307 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a481057-a054-4411-8e5a-5697f10b0ead None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:16.605567 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:16.688676 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a481057-a054-4411-8e5a-5697f10b0ead None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:16.720155 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62e2c6-c214-40cd-9412-e69dcbb551d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:16.720934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62e2c6-c214-40cd-9412-e69dcbb551d7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:16.721239 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62e2c6-c214-40cd-9412-e69dcbb551d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:16.721511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62e2c6-c214-40cd-9412-e69dcbb551d7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:16.728711 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62e2c6-c214-40cd-9412-e69dcbb551d7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:16.730176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62e2c6-c214-40cd-9412-e69dcbb551d7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:16.744043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-ce62e2c6-c214-40cd-9412-e69dcbb551d7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:21:16.744043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:16.744043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:16.744043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:16.744043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:16.744043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:16.744043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:16.744043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:16.744043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:16.744043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:16.744043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:16.744043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:16.744043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:21:16.744043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:21:16.744043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:21:16.744043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:21:16.745233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:16.745233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:16.745233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:21:16.745233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:21:16.745233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:21:16.745233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:21:16.745233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:16.745233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:16.745233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:16.745233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:16.745233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:16.745233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:16.745233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:16.745233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:16.745233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:16.745233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:16.745233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:16.745233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:16.746193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:16.746193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:16.746193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:21:16.746193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:21:16.746193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:21:16.746193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:21:16.746193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:21:16.746193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:21:16.746193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 08:21:16.746193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:16.746193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 547/1091] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Jun 7 08:21:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:16.751738 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214815f9-93e6-4929-81e2-dac67ac150ed None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:16.785051 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:16.848593 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214815f9-93e6-4929-81e2-dac67ac150ed None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:16.881342 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ad0fd-696b-40a7-912f-3efd9febf22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:16.882004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ad0fd-696b-40a7-912f-3efd9febf22b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:16.882283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ad0fd-696b-40a7-912f-3efd9febf22b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:16.882571 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ad0fd-696b-40a7-912f-3efd9febf22b None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:16.883390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ad0fd-696b-40a7-912f-3efd9febf22b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:16.883692 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ad0fd-696b-40a7-912f-3efd9febf22b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:16.885125 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ad0fd-696b-40a7-912f-3efd9febf22b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:16.901460 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 545/1092] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Jun 7 08:21:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:16.907089 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62e2c6-c214-40cd-9412-e69dcbb551d7 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:16.975298 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:17.004445 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62e2c6-c214-40cd-9412-e69dcbb551d7 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:17.038253 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915ec22-607e-416c-bf27-2ede378ac8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:17.038896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915ec22-607e-416c-bf27-2ede378ac8ad None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:17.039178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915ec22-607e-416c-bf27-2ede378ac8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:17.039454 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915ec22-607e-416c-bf27-2ede378ac8ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:17.105567 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915ec22-607e-416c-bf27-2ede378ac8ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:17.107506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915ec22-607e-416c-bf27-2ede378ac8ad None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:17.175132 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:17.203145 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 548/1093] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:21:17.208874 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ad0fd-696b-40a7-912f-3efd9febf22b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:17.298605 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:17.304893 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ad0fd-696b-40a7-912f-3efd9febf22b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:17.336508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9379a37-9751-47ed-b057-b14ad3888aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:17.337315 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9379a37-9751-47ed-b057-b14ad3888aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:17.337593 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9379a37-9751-47ed-b057-b14ad3888aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:17.337864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9379a37-9751-47ed-b057-b14ad3888aa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:17.347940 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9379a37-9751-47ed-b057-b14ad3888aa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:17.349466 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9379a37-9751-47ed-b057-b14ad3888aa9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:17.367548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a9379a37-9751-47ed-b057-b14ad3888aa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:17.367548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:17.367548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:17.367548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:17.367548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:17.367548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:17.367548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:17.367548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:17.367548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:17.367548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:17.367548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:17.367548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:17.367548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:21:17.367548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:21:17.367548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:21:17.368589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:21:17.368589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:17.368589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:17.368589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:17.368589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:17.368589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:17.368589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:17.368589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:17.368589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:17.368589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:17.368589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:17.368589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:17.368589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:17.368589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:17.368589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:17.368589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:17.368589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:17.370081 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:21:17.370081 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:21:17.370081 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:21:17.370081 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:17.370081 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 546/1094] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:21:17.374188 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915ec22-607e-416c-bf27-2ede378ac8ad None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:17.471165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915ec22-607e-416c-bf27-2ede378ac8ad None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:17.503129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3d537-97a8-4938-8ee1-50ece20e3a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:17.503776 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3d537-97a8-4938-8ee1-50ece20e3a40 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:17.504053 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3d537-97a8-4938-8ee1-50ece20e3a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:17.504321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3d537-97a8-4938-8ee1-50ece20e3a40 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:17.505052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3d537-97a8-4938-8ee1-50ece20e3a40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:17.505354 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3d537-97a8-4938-8ee1-50ece20e3a40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:17.506712 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3d537-97a8-4938-8ee1-50ece20e3a40 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:17.518073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 549/1095] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:17.523670 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9379a37-9751-47ed-b057-b14ad3888aa9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:17.619762 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9379a37-9751-47ed-b057-b14ad3888aa9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:17.651638 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b89c01-0440-4671-8fdf-217fa571a3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:17.652409 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b89c01-0440-4671-8fdf-217fa571a3ea None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:17.653249 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b89c01-0440-4671-8fdf-217fa571a3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:17.653519 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b89c01-0440-4671-8fdf-217fa571a3ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:17.659840 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b89c01-0440-4671-8fdf-217fa571a3ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:17.661302 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b89c01-0440-4671-8fdf-217fa571a3ea None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:17.674659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-c2b89c01-0440-4671-8fdf-217fa571a3ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:21:17.674659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:17.674659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:17.674659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:17.674659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:17.674659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:17.674659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:17.674659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:17.674659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:17.674659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:17.674659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:17.674659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:17.674659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:21:17.674659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:21:17.674659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:21:17.675734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:21:17.675734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:17.675734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:17.675734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:17.675734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:17.675734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:17.675734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:17.675734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:17.675734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:17.675734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:17.675734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:17.675734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:17.675734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:17.675734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:17.675734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:17.675734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:17.675734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:17.681869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:21:17.681869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:21:17.681869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:21:17.681869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:21:17.681869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:21:17.681869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:21:17.681869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:21:17.681869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:17.681869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 547/1096] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:21:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:17.682403 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3d537-97a8-4938-8ee1-50ece20e3a40 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:17.711980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:17.778866 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3d537-97a8-4938-8ee1-50ece20e3a40 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:17.811012 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe2e98-fe3f-47bb-899d-96b1f9206c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:17.811659 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe2e98-fe3f-47bb-899d-96b1f9206c22 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:17.811937 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe2e98-fe3f-47bb-899d-96b1f9206c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:17.812208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe2e98-fe3f-47bb-899d-96b1f9206c22 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:17.812982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe2e98-fe3f-47bb-899d-96b1f9206c22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:17.813313 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe2e98-fe3f-47bb-899d-96b1f9206c22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:17.814733 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe2e98-fe3f-47bb-899d-96b1f9206c22 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:17.826817 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 550/1097] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:21:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:17.833028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b89c01-0440-4671-8fdf-217fa571a3ea None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:17.842173 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:17.928629 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b89c01-0440-4671-8fdf-217fa571a3ea None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:17.961597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecb285-3be0-418e-a308-e73888852ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:17.962254 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecb285-3be0-418e-a308-e73888852ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:17.962538 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecb285-3be0-418e-a308-e73888852ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:17.962817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecb285-3be0-418e-a308-e73888852ca2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:17.963640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ecb285-3be0-418e-a308-e73888852ca2 None admin] RBAC: Adding query filter params (role.id=c782c67dcd6a46cc8c5b26a87d6d2dc3, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=ee42568b059b49a198ed2c01d5efdafd) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:17.963953 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ecb285-3be0-418e-a308-e73888852ca2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:17.965439 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ecb285-3be0-418e-a308-e73888852ca2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:17.971146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 548/1098] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:17 2022] GET /identity/v3/role_assignments?user.id=ee42568b059b49a198ed2c01d5efdafd&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=c782c67dcd6a46cc8c5b26a87d6d2dc3 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:18.562591 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff63d0d-27c7-4e46-98f7-ff43d8873570 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:18.563100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff63d0d-27c7-4e46-98f7-ff43d8873570 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:18.563100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff63d0d-27c7-4e46-98f7-ff43d8873570 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:18.564680 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 551/1099] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:18.570023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e08e4-0c6e-450c-a1cc-da8a9f3dc054 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:18.570446 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e08e4-0c6e-450c-a1cc-da8a9f3dc054 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:18.570586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e08e4-0c6e-450c-a1cc-da8a9f3dc054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:18.587479 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a87e08e4-0c6e-450c-a1cc-da8a9f3dc054 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:18.588672 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a87e08e4-0c6e-450c-a1cc-da8a9f3dc054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:18.651279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:18.656394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 549/1100] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:18.681356 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff63d0d-27c7-4e46-98f7-ff43d8873570 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:18.776478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:18.786514 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff63d0d-27c7-4e46-98f7-ff43d8873570 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:18.821886 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79e102-bd1b-4a93-bf78-94069b4ffa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:18.822601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79e102-bd1b-4a93-bf78-94069b4ffa50 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:18.822884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79e102-bd1b-4a93-bf78-94069b4ffa50 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:18.823158 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79e102-bd1b-4a93-bf78-94069b4ffa50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:18.833305 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79e102-bd1b-4a93-bf78-94069b4ffa50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:18.841347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-ac79e102-bd1b-4a93-bf78-94069b4ffa50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:18.841347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:18.841347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:18.841347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:18.841347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:18.841347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:18.841347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:18.841347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:18.841347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:18.841347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:18.841347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:18.841347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:18.841347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:18.841347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:18.841347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:18.841347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:18.842347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:18.842347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:18.842347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:18.842347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:18.842347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:18.842347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:18.842347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:18.842347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:18.842347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:18.842347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:18.842347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:18.842347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:18.842347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:18.842347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:18.842347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:18.842347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:18.843197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:18.843197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:18.843197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:18.843197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:18.843197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:18.843197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:18.843197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:18.843197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:18.843197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:18.843197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:18.843197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:18.843197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:18.843197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 552/1101] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:21:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:18.849210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e08e4-0c6e-450c-a1cc-da8a9f3dc054 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:18.943767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e08e4-0c6e-450c-a1cc-da8a9f3dc054 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:18.976605 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2afd1-f7e2-45a6-a4eb-383c58ba276f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:18.977289 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2afd1-f7e2-45a6-a4eb-383c58ba276f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:18.977573 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2afd1-f7e2-45a6-a4eb-383c58ba276f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:18.977848 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2afd1-f7e2-45a6-a4eb-383c58ba276f None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:18.978587 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2afd1-f7e2-45a6-a4eb-383c58ba276f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:18.978896 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2afd1-f7e2-45a6-a4eb-383c58ba276f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:18.980252 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2afd1-f7e2-45a6-a4eb-383c58ba276f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:18.988037 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 550/1102] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:18.993666 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79e102-bd1b-4a93-bf78-94069b4ffa50 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:19.091528 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79e102-bd1b-4a93-bf78-94069b4ffa50 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:19.122919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e876f3-5eec-489b-8564-9343d3bccced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:19.123767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e876f3-5eec-489b-8564-9343d3bccced None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:19.124049 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e876f3-5eec-489b-8564-9343d3bccced None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:19.124323 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e876f3-5eec-489b-8564-9343d3bccced None admin] PATH_INFO: `/v3/users/ac218165cb9a4b47a216d9f53592c242` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:19.125505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e876f3-5eec-489b-8564-9343d3bccced None admin] RBAC: Authorizing `identity:get_user(user_id=ac218165cb9a4b47a216d9f53592c242)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:19.126942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e876f3-5eec-489b-8564-9343d3bccced None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:19.128522 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 553/1103] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:18 2022] GET /identity/v3/users/ac218165cb9a4b47a216d9f53592c242 => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:19.133981 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2afd1-f7e2-45a6-a4eb-383c58ba276f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:19.193976 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:19.231096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2afd1-f7e2-45a6-a4eb-383c58ba276f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:19.263962 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62fa8a5-a2c4-4937-a103-4ba2e4355b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:19.264602 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fa8a5-a2c4-4937-a103-4ba2e4355b57 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:19.264887 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fa8a5-a2c4-4937-a103-4ba2e4355b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:19.265192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fa8a5-a2c4-4937-a103-4ba2e4355b57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:19.329129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62fa8a5-a2c4-4937-a103-4ba2e4355b57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:19.330582 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62fa8a5-a2c4-4937-a103-4ba2e4355b57 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:19.386335 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:19.423765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 551/1104] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:21:19.429725 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e876f3-5eec-489b-8564-9343d3bccced None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:19.452079 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:19.525834 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e876f3-5eec-489b-8564-9343d3bccced None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:19.557215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b7795-e5fe-4e6d-8205-e93b4d590c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:19.558051 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b7795-e5fe-4e6d-8205-e93b4d590c70 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:19.558325 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b7795-e5fe-4e6d-8205-e93b4d590c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:19.558590 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b7795-e5fe-4e6d-8205-e93b4d590c70 None admin] PATH_INFO: `/v3/projects/6a50c552a3034a138ae711d82c59f9f9` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:19.560276 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b7795-e5fe-4e6d-8205-e93b4d590c70 None admin] RBAC: Authorizing `identity:get_project(project_id=6a50c552a3034a138ae711d82c59f9f9)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:19.561726 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b7795-e5fe-4e6d-8205-e93b4d590c70 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:19.563360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 554/1105] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:19 2022] GET /identity/v3/projects/6a50c552a3034a138ae711d82c59f9f9 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:19.569296 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62fa8a5-a2c4-4937-a103-4ba2e4355b57 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:19.663351 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62fa8a5-a2c4-4937-a103-4ba2e4355b57 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:19.695986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed475e3-5dce-441d-ae4c-a211706f1d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:19.696620 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed475e3-5dce-441d-ae4c-a211706f1d78 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:19.696896 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed475e3-5dce-441d-ae4c-a211706f1d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:19.697195 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed475e3-5dce-441d-ae4c-a211706f1d78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:19.698005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed475e3-5dce-441d-ae4c-a211706f1d78 None admin] RBAC: Adding query filter params (role.id=8c63c00418a346b29f7db3f87a404e70, scope.project.id=6a50c552a3034a138ae711d82c59f9f9, user.id=ac218165cb9a4b47a216d9f53592c242) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:19.698313 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed475e3-5dce-441d-ae4c-a211706f1d78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:19.699742 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed475e3-5dce-441d-ae4c-a211706f1d78 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:19.705262 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 552/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:19 2022] GET /identity/v3/role_assignments?user.id=ac218165cb9a4b47a216d9f53592c242&scope.project.id=6a50c552a3034a138ae711d82c59f9f9&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:21.931690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6925bc7-1564-4c17-bb71-ef9410a7f5fa None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:21.932217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6925bc7-1564-4c17-bb71-ef9410a7f5fa None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:21.932217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6925bc7-1564-4c17-bb71-ef9410a7f5fa None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:21.933618 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 555/1107] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:21.938822 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e8c4e-11b0-4ef9-a89e-d1384f99d350 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:21.939226 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e8c4e-11b0-4ef9-a89e-d1384f99d350 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:21.939384 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e8c4e-11b0-4ef9-a89e-d1384f99d350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:21.956161 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3a0e8c4e-11b0-4ef9-a89e-d1384f99d350 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:21.957372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3a0e8c4e-11b0-4ef9-a89e-d1384f99d350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:21.977115 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:22.020727 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 553/1108] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:22.045545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6925bc7-1564-4c17-bb71-ef9410a7f5fa None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:22.130237 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:22.142845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6925bc7-1564-4c17-bb71-ef9410a7f5fa None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:22.175466 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89dc34-be12-4544-8add-6d9d4a390b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:22.176257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89dc34-be12-4544-8add-6d9d4a390b30 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:22.176535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89dc34-be12-4544-8add-6d9d4a390b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:22.176803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89dc34-be12-4544-8add-6d9d4a390b30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:22.177986 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89dc34-be12-4544-8add-6d9d4a390b30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:22.179406 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89dc34-be12-4544-8add-6d9d4a390b30 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:22.180977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 556/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:22.186779 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e8c4e-11b0-4ef9-a89e-d1384f99d350 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:22.282990 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e8c4e-11b0-4ef9-a89e-d1384f99d350 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:22.315468 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb28148-80ce-4803-872e-ea805f553e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:22.316123 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb28148-80ce-4803-872e-ea805f553e0f None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:22.316410 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb28148-80ce-4803-872e-ea805f553e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:22.316689 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb28148-80ce-4803-872e-ea805f553e0f None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:22.317419 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb28148-80ce-4803-872e-ea805f553e0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:22.318841 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb28148-80ce-4803-872e-ea805f553e0f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:22.400294 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 554/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jun 7 08:21:22 2022] POST /identity/v3/users => generated 286 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:21:24.649735 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e728e-76ba-459b-9e64-4f7f0481e70e None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:24.650253 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e728e-76ba-459b-9e64-4f7f0481e70e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:24.650253 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e728e-76ba-459b-9e64-4f7f0481e70e None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:24.651550 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 557/1111] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:24.657415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d66eb6-91e6-4039-a717-f3567ba25373 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:24.657809 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d66eb6-91e6-4039-a717-f3567ba25373 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:24.657974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d66eb6-91e6-4039-a717-f3567ba25373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:24.675117 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-55d66eb6-91e6-4039-a717-f3567ba25373 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:24.676290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-55d66eb6-91e6-4039-a717-f3567ba25373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:24.683336 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:24.740891 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 555/1112] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:24.763853 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e728e-76ba-459b-9e64-4f7f0481e70e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:24.862484 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e728e-76ba-459b-9e64-4f7f0481e70e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:24.893979 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293de26-e3e5-4a1d-b538-0f33bd5fb9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:24.894811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293de26-e3e5-4a1d-b538-0f33bd5fb9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:24.895086 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293de26-e3e5-4a1d-b538-0f33bd5fb9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:24.895352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293de26-e3e5-4a1d-b538-0f33bd5fb9e2 None admin] PATH_INFO: `/v3/roles/4552e69b96eb43f3ba6a9ca694d04ffb` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:24.896781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293de26-e3e5-4a1d-b538-0f33bd5fb9e2 None admin] RBAC: Authorizing `identity:get_role(role_id=4552e69b96eb43f3ba6a9ca694d04ffb)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:24.898167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293de26-e3e5-4a1d-b538-0f33bd5fb9e2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:24.899498 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 558/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:24 2022] GET /identity/v3/roles/4552e69b96eb43f3ba6a9ca694d04ffb => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:24.905601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d66eb6-91e6-4039-a717-f3567ba25373 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:24.998570 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d66eb6-91e6-4039-a717-f3567ba25373 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:25.030941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f34c4-8cca-4a00-948a-8f78993cde4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:25.031785 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f34c4-8cca-4a00-948a-8f78993cde4b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:25.032061 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f34c4-8cca-4a00-948a-8f78993cde4b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:25.032330 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f34c4-8cca-4a00-948a-8f78993cde4b None admin] PATH_INFO: `/v3/users/a98faa9b20be4a40a75ceaad488d58b0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:25.043367 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:25.045598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f34c4-8cca-4a00-948a-8f78993cde4b None admin] RBAC: Authorizing `identity:get_user(user_id=a98faa9b20be4a40a75ceaad488d58b0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:25.047062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f34c4-8cca-4a00-948a-8f78993cde4b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:25.048968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 556/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:24 2022] GET /identity/v3/users/a98faa9b20be4a40a75ceaad488d58b0 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:25.054566 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293de26-e3e5-4a1d-b538-0f33bd5fb9e2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:25.079723 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:25.151928 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293de26-e3e5-4a1d-b538-0f33bd5fb9e2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:25.184758 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f830367-9eb5-4875-b52f-48040c1577c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:25.185426 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f830367-9eb5-4875-b52f-48040c1577c9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:25.185714 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f830367-9eb5-4875-b52f-48040c1577c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:25.185991 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f830367-9eb5-4875-b52f-48040c1577c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:25.251645 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f830367-9eb5-4875-b52f-48040c1577c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:25.253179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f830367-9eb5-4875-b52f-48040c1577c9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:25.276468 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:25.349486 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 559/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:21:25.355463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f34c4-8cca-4a00-948a-8f78993cde4b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:25.455800 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f34c4-8cca-4a00-948a-8f78993cde4b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:25.487448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29645d8-22cc-437b-ac52-34d9959e157e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:25.488290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29645d8-22cc-437b-ac52-34d9959e157e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:25.488564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29645d8-22cc-437b-ac52-34d9959e157e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:25.488833 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29645d8-22cc-437b-ac52-34d9959e157e None admin] PATH_INFO: `/v3/projects/6a50c552a3034a138ae711d82c59f9f9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:25.489998 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29645d8-22cc-437b-ac52-34d9959e157e None admin] RBAC: Authorizing `identity:get_project(project_id=6a50c552a3034a138ae711d82c59f9f9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:25.491397 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29645d8-22cc-437b-ac52-34d9959e157e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:25.493048 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 557/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:25 2022] GET /identity/v3/projects/6a50c552a3034a138ae711d82c59f9f9 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:25.499424 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f830367-9eb5-4875-b52f-48040c1577c9 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:25.593436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f830367-9eb5-4875-b52f-48040c1577c9 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:25.615856 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:25.626540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93199beb-3fa5-4902-9eec-1dcaf48cf155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:25.627170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93199beb-3fa5-4902-9eec-1dcaf48cf155 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:25.627447 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93199beb-3fa5-4902-9eec-1dcaf48cf155 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:25.627719 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93199beb-3fa5-4902-9eec-1dcaf48cf155 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:25.630738 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93199beb-3fa5-4902-9eec-1dcaf48cf155 None admin] RBAC: Adding query filter params (role.id=4552e69b96eb43f3ba6a9ca694d04ffb, scope.project.id=6a50c552a3034a138ae711d82c59f9f9, user.id=a98faa9b20be4a40a75ceaad488d58b0) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:25.630738 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93199beb-3fa5-4902-9eec-1dcaf48cf155 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:25.630738 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93199beb-3fa5-4902-9eec-1dcaf48cf155 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:25.635487 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 560/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:25 2022] GET /identity/v3/role_assignments?user.id=a98faa9b20be4a40a75ceaad488d58b0&scope.project.id=6a50c552a3034a138ae711d82c59f9f9&role.id=4552e69b96eb43f3ba6a9ca694d04ffb => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:27.812854 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08d280-87e8-4cd0-939b-d1026c1cd612 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:27.813410 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08d280-87e8-4cd0-939b-d1026c1cd612 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:27.813528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08d280-87e8-4cd0-939b-d1026c1cd612 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:27.814937 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 558/1118] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:27.820837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a0f0d-8bd8-4e14-b19a-477c1b4eb758 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:27.821295 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a0f0d-8bd8-4e14-b19a-477c1b4eb758 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:27.821417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a0f0d-8bd8-4e14-b19a-477c1b4eb758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:27.837536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-cb7a0f0d-8bd8-4e14-b19a-477c1b4eb758 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:27.838709 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cb7a0f0d-8bd8-4e14-b19a-477c1b4eb758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:27.904690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 561/1119] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:27.927504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08d280-87e8-4cd0-939b-d1026c1cd612 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:27.993016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:28.025949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08d280-87e8-4cd0-939b-d1026c1cd612 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:28.058683 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049285a7-8c56-42bc-8ae3-b2abf0822124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:28.059525 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049285a7-8c56-42bc-8ae3-b2abf0822124 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:28.059810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049285a7-8c56-42bc-8ae3-b2abf0822124 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:28.060086 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049285a7-8c56-42bc-8ae3-b2abf0822124 None admin] PATH_INFO: `/v3/roles/4552e69b96eb43f3ba6a9ca694d04ffb` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:28.061578 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049285a7-8c56-42bc-8ae3-b2abf0822124 None admin] RBAC: Authorizing `identity:get_role(role_id=4552e69b96eb43f3ba6a9ca694d04ffb)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:28.062986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049285a7-8c56-42bc-8ae3-b2abf0822124 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:28.064274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 559/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:27 2022] GET /identity/v3/roles/4552e69b96eb43f3ba6a9ca694d04ffb => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:28.069868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a0f0d-8bd8-4e14-b19a-477c1b4eb758 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:28.164517 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a0f0d-8bd8-4e14-b19a-477c1b4eb758 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:28.181271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:28.197434 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafff1e8-54e9-4aae-89d2-9d52230a4963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:28.198294 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafff1e8-54e9-4aae-89d2-9d52230a4963 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:28.198569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafff1e8-54e9-4aae-89d2-9d52230a4963 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:28.198837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafff1e8-54e9-4aae-89d2-9d52230a4963 None admin] PATH_INFO: `/v3/users/a98faa9b20be4a40a75ceaad488d58b0` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:28.200498 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafff1e8-54e9-4aae-89d2-9d52230a4963 None admin] RBAC: Authorizing `identity:get_user(user_id=a98faa9b20be4a40a75ceaad488d58b0)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:28.201946 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafff1e8-54e9-4aae-89d2-9d52230a4963 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:28.203540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 562/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:28 2022] GET /identity/v3/users/a98faa9b20be4a40a75ceaad488d58b0 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:28.209196 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049285a7-8c56-42bc-8ae3-b2abf0822124 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:28.303874 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049285a7-8c56-42bc-8ae3-b2abf0822124 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:28.335987 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92898cb8-b61d-4964-a915-308efde48e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:28.336767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92898cb8-b61d-4964-a915-308efde48e03 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:28.337072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92898cb8-b61d-4964-a915-308efde48e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:28.337347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92898cb8-b61d-4964-a915-308efde48e03 None admin] PATH_INFO: `/v3/projects/6a50c552a3034a138ae711d82c59f9f9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:28.338497 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92898cb8-b61d-4964-a915-308efde48e03 None admin] RBAC: Authorizing `identity:get_project(project_id=6a50c552a3034a138ae711d82c59f9f9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:28.339930 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92898cb8-b61d-4964-a915-308efde48e03 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:28.341608 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 560/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:28 2022] GET /identity/v3/projects/6a50c552a3034a138ae711d82c59f9f9 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:28.347226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafff1e8-54e9-4aae-89d2-9d52230a4963 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:28.442049 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafff1e8-54e9-4aae-89d2-9d52230a4963 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:28.474313 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25334459-ef11-49b2-82f5-1fac133f97a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:28.475175 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25334459-ef11-49b2-82f5-1fac133f97a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:28.475454 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25334459-ef11-49b2-82f5-1fac133f97a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:28.475727 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25334459-ef11-49b2-82f5-1fac133f97a0 None admin] PATH_INFO: `/v3/projects/6a50c552a3034a138ae711d82c59f9f9/users/a98faa9b20be4a40a75ceaad488d58b0/roles/4552e69b96eb43f3ba6a9ca694d04ffb` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:28.477893 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25334459-ef11-49b2-82f5-1fac133f97a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a50c552a3034a138ae711d82c59f9f9, user_id=a98faa9b20be4a40a75ceaad488d58b0, role_id=4552e69b96eb43f3ba6a9ca694d04ffb)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:28.479302 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25334459-ef11-49b2-82f5-1fac133f97a0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:28.559782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 563/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:21:28 2022] PUT /identity/v3/projects/6a50c552a3034a138ae711d82c59f9f9/users/a98faa9b20be4a40a75ceaad488d58b0/roles/4552e69b96eb43f3ba6a9ca694d04ffb => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:21:30.751140 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774ab85-a002-4437-b413-55a9a51261ae None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:30.751675 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774ab85-a002-4437-b413-55a9a51261ae None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:30.751675 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774ab85-a002-4437-b413-55a9a51261ae None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:30.753071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 561/1124] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:30.758141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9995ccd-8775-4659-b230-e84cf9b5a32d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:30.758573 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9995ccd-8775-4659-b230-e84cf9b5a32d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:30.758672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9995ccd-8775-4659-b230-e84cf9b5a32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:30.775513 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a9995ccd-8775-4659-b230-e84cf9b5a32d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:30.776674 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a9995ccd-8775-4659-b230-e84cf9b5a32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:30.816463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:30.840022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 564/1125] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:30.863029 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774ab85-a002-4437-b413-55a9a51261ae None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:30.866702 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:30.960642 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774ab85-a002-4437-b413-55a9a51261ae None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:30.992583 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e8d72-b78a-4c9d-a4ff-c02502866035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:30.993446 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e8d72-b78a-4c9d-a4ff-c02502866035 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:30.993723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e8d72-b78a-4c9d-a4ff-c02502866035 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:30.993988 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e8d72-b78a-4c9d-a4ff-c02502866035 None admin] PATH_INFO: `/v3/roles/4552e69b96eb43f3ba6a9ca694d04ffb` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:30.995422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e8d72-b78a-4c9d-a4ff-c02502866035 None admin] RBAC: Authorizing `identity:get_role(role_id=4552e69b96eb43f3ba6a9ca694d04ffb)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:30.996797 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e8d72-b78a-4c9d-a4ff-c02502866035 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:30.998141 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 562/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:30 2022] GET /identity/v3/roles/4552e69b96eb43f3ba6a9ca694d04ffb => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:31.003761 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9995ccd-8775-4659-b230-e84cf9b5a32d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:31.097816 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9995ccd-8775-4659-b230-e84cf9b5a32d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:31.129903 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d660c2-59ae-477d-98d0-6aa77b57f40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:31.130761 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d660c2-59ae-477d-98d0-6aa77b57f40a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:31.131044 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d660c2-59ae-477d-98d0-6aa77b57f40a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:31.131317 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d660c2-59ae-477d-98d0-6aa77b57f40a None admin] PATH_INFO: `/v3/users/a98faa9b20be4a40a75ceaad488d58b0` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:31.132478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d660c2-59ae-477d-98d0-6aa77b57f40a None admin] RBAC: Authorizing `identity:get_user(user_id=a98faa9b20be4a40a75ceaad488d58b0)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:31.133947 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d660c2-59ae-477d-98d0-6aa77b57f40a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:31.135521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 565/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:31 2022] GET /identity/v3/users/a98faa9b20be4a40a75ceaad488d58b0 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:31.141442 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e8d72-b78a-4c9d-a4ff-c02502866035 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:31.208740 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:31.236828 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e8d72-b78a-4c9d-a4ff-c02502866035 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:31.269606 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc83514-6d43-4012-95e7-9c0035fd5e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:31.270183 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc83514-6d43-4012-95e7-9c0035fd5e35 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:31.270466 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc83514-6d43-4012-95e7-9c0035fd5e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:31.270743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc83514-6d43-4012-95e7-9c0035fd5e35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:31.334635 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc83514-6d43-4012-95e7-9c0035fd5e35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:31.336078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc83514-6d43-4012-95e7-9c0035fd5e35 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:31.399554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:31.427867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 563/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:21:31.434217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d660c2-59ae-477d-98d0-6aa77b57f40a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:31.477836 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:31.529654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d660c2-59ae-477d-98d0-6aa77b57f40a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:31.561780 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829a13f-c666-4ad5-a027-fe33f159ec0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:31.562618 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829a13f-c666-4ad5-a027-fe33f159ec0f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:31.562887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829a13f-c666-4ad5-a027-fe33f159ec0f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:31.563156 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829a13f-c666-4ad5-a027-fe33f159ec0f None admin] PATH_INFO: `/v3/projects/6a50c552a3034a138ae711d82c59f9f9` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:31.564298 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829a13f-c666-4ad5-a027-fe33f159ec0f None admin] RBAC: Authorizing `identity:get_project(project_id=6a50c552a3034a138ae711d82c59f9f9)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:31.565735 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829a13f-c666-4ad5-a027-fe33f159ec0f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:31.567353 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 566/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:31 2022] GET /identity/v3/projects/6a50c552a3034a138ae711d82c59f9f9 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:31.573279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc83514-6d43-4012-95e7-9c0035fd5e35 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:31.665611 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc83514-6d43-4012-95e7-9c0035fd5e35 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:31.698091 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f6ee4f-261f-470b-b21e-9dcb661c089c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:31.698720 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6ee4f-261f-470b-b21e-9dcb661c089c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:31.698995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6ee4f-261f-470b-b21e-9dcb661c089c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:31.699287 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6ee4f-261f-470b-b21e-9dcb661c089c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:31.700095 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f6ee4f-261f-470b-b21e-9dcb661c089c None admin] RBAC: Adding query filter params (role.id=4552e69b96eb43f3ba6a9ca694d04ffb, scope.project.id=6a50c552a3034a138ae711d82c59f9f9, user.id=a98faa9b20be4a40a75ceaad488d58b0) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:31.700400 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f6ee4f-261f-470b-b21e-9dcb661c089c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:31.701824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f6ee4f-261f-470b-b21e-9dcb661c089c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:31.707304 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 564/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:31 2022] GET /identity/v3/role_assignments?user.id=a98faa9b20be4a40a75ceaad488d58b0&scope.project.id=6a50c552a3034a138ae711d82c59f9f9&role.id=4552e69b96eb43f3ba6a9ca694d04ffb => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:33.943639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc83dc-3e70-4e55-b866-949615e856f1 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:33.944203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc83dc-3e70-4e55-b866-949615e856f1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:33.944203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc83dc-3e70-4e55-b866-949615e856f1 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:33.945592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 567/1131] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:33.950954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681e2f0-ca00-435c-b005-0e32b251629a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:33.951369 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681e2f0-ca00-435c-b005-0e32b251629a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:33.951515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681e2f0-ca00-435c-b005-0e32b251629a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:33.967763 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3681e2f0-ca00-435c-b005-0e32b251629a None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:33.968933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3681e2f0-ca00-435c-b005-0e32b251629a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:33.997123 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:34.032661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 565/1132] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:21:34.056560 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc83dc-3e70-4e55-b866-949615e856f1 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:34.151752 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc83dc-3e70-4e55-b866-949615e856f1 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:34.164083 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:34.185366 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a5555-7533-48ea-8d0c-6780ae0f032a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:34.186142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a5555-7533-48ea-8d0c-6780ae0f032a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:34.186418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a5555-7533-48ea-8d0c-6780ae0f032a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:34.186685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a5555-7533-48ea-8d0c-6780ae0f032a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:34.187888 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a5555-7533-48ea-8d0c-6780ae0f032a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:34.189664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a5555-7533-48ea-8d0c-6780ae0f032a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:34.191676 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 568/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:34.197267 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3681e2f0-ca00-435c-b005-0e32b251629a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:34.290399 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3681e2f0-ca00-435c-b005-0e32b251629a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:34.322697 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b781077-5b82-453b-9fac-bb3604736765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:34.323337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b781077-5b82-453b-9fac-bb3604736765 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:34.323619 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b781077-5b82-453b-9fac-bb3604736765 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:34.323894 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b781077-5b82-453b-9fac-bb3604736765 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:34.324591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b781077-5b82-453b-9fac-bb3604736765 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:34.326039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b781077-5b82-453b-9fac-bb3604736765 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:34.412267 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 566/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jun 7 08:21:34 2022] POST /identity/v3/projects => generated 311 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:21:36.677084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f06aa-96ae-4109-9d27-3ab376272be4 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:36.677619 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f06aa-96ae-4109-9d27-3ab376272be4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:36.677723 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f06aa-96ae-4109-9d27-3ab376272be4 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:36.679071 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 569/1135] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:36.684962 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c0879-1f5e-413d-8fbf-bd2ec415ef9e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:36.685360 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c0879-1f5e-413d-8fbf-bd2ec415ef9e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:36.685545 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c0879-1f5e-413d-8fbf-bd2ec415ef9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:36.702535 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a91c0879-1f5e-413d-8fbf-bd2ec415ef9e None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:36.703716 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a91c0879-1f5e-413d-8fbf-bd2ec415ef9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:36.716427 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:36.767641 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 567/1136] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:36.792589 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f06aa-96ae-4109-9d27-3ab376272be4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:36.890089 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f06aa-96ae-4109-9d27-3ab376272be4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:36.921922 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f028271-3fe3-4cc3-9ac1-210752bc9f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:36.922680 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f028271-3fe3-4cc3-9ac1-210752bc9f21 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:36.923211 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f028271-3fe3-4cc3-9ac1-210752bc9f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:36.923478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f028271-3fe3-4cc3-9ac1-210752bc9f21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:36.924620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f028271-3fe3-4cc3-9ac1-210752bc9f21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:36.926068 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f028271-3fe3-4cc3-9ac1-210752bc9f21 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:36.927659 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 570/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:21:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:36.933091 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c0879-1f5e-413d-8fbf-bd2ec415ef9e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:37.026261 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c0879-1f5e-413d-8fbf-bd2ec415ef9e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:37.059777 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe09a4-6b35-4ed6-9e4f-5c808627e342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:37.060428 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe09a4-6b35-4ed6-9e4f-5c808627e342 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:37.060707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe09a4-6b35-4ed6-9e4f-5c808627e342 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:37.060978 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe09a4-6b35-4ed6-9e4f-5c808627e342 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:37.061700 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe09a4-6b35-4ed6-9e4f-5c808627e342 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:37.063101 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe09a4-6b35-4ed6-9e4f-5c808627e342 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:37.165319 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 568/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jun 7 08:21:36 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:21:39.434127 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589989f0-6310-4cda-b124-0a9516e39d64 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:39.434676 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589989f0-6310-4cda-b124-0a9516e39d64 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:39.434676 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589989f0-6310-4cda-b124-0a9516e39d64 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:39.436035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 571/1139] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:39.441398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d42bca-1b15-4698-b021-7cee0ea3c99d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:39.441806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d42bca-1b15-4698-b021-7cee0ea3c99d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:39.441909 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d42bca-1b15-4698-b021-7cee0ea3c99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:39.447693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:39.458182 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-58d42bca-1b15-4698-b021-7cee0ea3c99d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:39.459363 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-58d42bca-1b15-4698-b021-7cee0ea3c99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:39.523709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 569/1140] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:21:39.548234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589989f0-6310-4cda-b124-0a9516e39d64 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:39.595921 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:39.644555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589989f0-6310-4cda-b124-0a9516e39d64 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:39.677355 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9b310-d63d-4c8b-9492-b322ed5ad1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:39.678118 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9b310-d63d-4c8b-9492-b322ed5ad1ab None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:39.678395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9b310-d63d-4c8b-9492-b322ed5ad1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:39.678663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9b310-d63d-4c8b-9492-b322ed5ad1ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:39.688248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9b310-d63d-4c8b-9492-b322ed5ad1ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:39.695744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-61d9b310-d63d-4c8b-9492-b322ed5ad1ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:39.695744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:39.695744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:39.695744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:39.695744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:39.695744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:39.695744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:39.695744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:39.695744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:39.695744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:39.695744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:39.695744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:39.695744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:39.695744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:39.695744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:39.695744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:39.696840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:39.696840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:39.696840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:39.696840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:39.696840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:39.696840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:39.696840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:39.696840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:39.696840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:39.696840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:39.696840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:39.696840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:39.696840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:39.696840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:39.696840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:39.696840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:39.697757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:39.697757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:39.697757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:39.697757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:39.697757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:39.697757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:39.697757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:39.697757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:39.697757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:39.697757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:39.697757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:39.697757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:39.697757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 572/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:21:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:39.702765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d42bca-1b15-4698-b021-7cee0ea3c99d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:39.796723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d42bca-1b15-4698-b021-7cee0ea3c99d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:39.828725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d519e-36b6-4180-958b-4eb572d21a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:39.829401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d519e-36b6-4180-958b-4eb572d21a22 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:39.829683 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d519e-36b6-4180-958b-4eb572d21a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:39.829952 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d519e-36b6-4180-958b-4eb572d21a22 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:39.830681 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d519e-36b6-4180-958b-4eb572d21a22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:39.830983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d519e-36b6-4180-958b-4eb572d21a22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:39.832339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d519e-36b6-4180-958b-4eb572d21a22 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:39.833669 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:39.841236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 570/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:21:39.847022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9b310-d63d-4c8b-9492-b322ed5ad1ab None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:39.945488 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9b310-d63d-4c8b-9492-b322ed5ad1ab None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:39.954595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:39.978609 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af785af2-c510-4917-853f-5766888f6ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:39.979479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af785af2-c510-4917-853f-5766888f6ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:39.979766 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af785af2-c510-4917-853f-5766888f6ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:39.980042 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af785af2-c510-4917-853f-5766888f6ad0 None admin] PATH_INFO: `/v3/users/68909600b94c4291862dc76586ed9a00` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:39.993242 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af785af2-c510-4917-853f-5766888f6ad0 None admin] RBAC: Authorizing `identity:get_user(user_id=68909600b94c4291862dc76586ed9a00)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:39.994680 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af785af2-c510-4917-853f-5766888f6ad0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:39.996311 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 573/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:39 2022] GET /identity/v3/users/68909600b94c4291862dc76586ed9a00 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:40.002035 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d519e-36b6-4180-958b-4eb572d21a22 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:40.097348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d519e-36b6-4180-958b-4eb572d21a22 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:40.128472 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a632b-6ba7-4072-8d05-42c2f5d3ca71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:40.129136 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a632b-6ba7-4072-8d05-42c2f5d3ca71 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:40.129417 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a632b-6ba7-4072-8d05-42c2f5d3ca71 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:40.129692 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a632b-6ba7-4072-8d05-42c2f5d3ca71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:40.187763 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:40.194107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a632b-6ba7-4072-8d05-42c2f5d3ca71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:40.195547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a632b-6ba7-4072-8d05-42c2f5d3ca71 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:40.289334 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 571/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:21:40.295163 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af785af2-c510-4917-853f-5766888f6ad0 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:40.391307 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af785af2-c510-4917-853f-5766888f6ad0 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:40.423199 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b65ab4-086a-436b-abe9-593f43cfc39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:40.424043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b65ab4-086a-436b-abe9-593f43cfc39c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:40.424324 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b65ab4-086a-436b-abe9-593f43cfc39c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:40.424598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b65ab4-086a-436b-abe9-593f43cfc39c None admin] PATH_INFO: `/v3/projects/643922019b474c089b72b74ccfc30675` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:40.425783 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b65ab4-086a-436b-abe9-593f43cfc39c None admin] RBAC: Authorizing `identity:get_project(project_id=643922019b474c089b72b74ccfc30675)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:40.427206 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b65ab4-086a-436b-abe9-593f43cfc39c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:40.428850 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 574/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:40 2022] GET /identity/v3/projects/643922019b474c089b72b74ccfc30675 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:40.435222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a632b-6ba7-4072-8d05-42c2f5d3ca71 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:40.523385 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:40.528835 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a632b-6ba7-4072-8d05-42c2f5d3ca71 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:40.561085 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d1b46-c001-4205-b9a9-eb6c60a3cfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:40.561671 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d1b46-c001-4205-b9a9-eb6c60a3cfb9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:40.561953 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d1b46-c001-4205-b9a9-eb6c60a3cfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:40.562228 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d1b46-c001-4205-b9a9-eb6c60a3cfb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:40.563047 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d1b46-c001-4205-b9a9-eb6c60a3cfb9 None admin] RBAC: Adding query filter params (role.id=8c63c00418a346b29f7db3f87a404e70, scope.project.id=643922019b474c089b72b74ccfc30675, user.id=68909600b94c4291862dc76586ed9a00) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:40.563363 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d1b46-c001-4205-b9a9-eb6c60a3cfb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:40.564775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d1b46-c001-4205-b9a9-eb6c60a3cfb9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:40.569983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 572/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:40 2022] GET /identity/v3/role_assignments?user.id=68909600b94c4291862dc76586ed9a00&scope.project.id=643922019b474c089b72b74ccfc30675&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:42.768254 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109964e-1cc0-4628-9489-d1f843d21f5e None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:42.768811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109964e-1cc0-4628-9489-d1f843d21f5e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:42.768811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109964e-1cc0-4628-9489-d1f843d21f5e None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:42.770180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 575/1147] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:42.775413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1716d29-d68f-43d8-8d88-f6b5dc8c51e4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:42.775811 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1716d29-d68f-43d8-8d88-f6b5dc8c51e4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:42.775973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1716d29-d68f-43d8-8d88-f6b5dc8c51e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:42.793923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d1716d29-d68f-43d8-8d88-f6b5dc8c51e4 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:42.795105 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d1716d29-d68f-43d8-8d88-f6b5dc8c51e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:42.858580 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 573/1148] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:42.880402 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109964e-1cc0-4628-9489-d1f843d21f5e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:42.914729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:42.978250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109964e-1cc0-4628-9489-d1f843d21f5e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:43.009649 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133ffc5-be12-46de-ad52-70338537b43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:43.010414 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133ffc5-be12-46de-ad52-70338537b43c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:43.010690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133ffc5-be12-46de-ad52-70338537b43c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:43.010961 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133ffc5-be12-46de-ad52-70338537b43c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:43.021053 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133ffc5-be12-46de-ad52-70338537b43c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:43.029011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-7133ffc5-be12-46de-ad52-70338537b43c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:43.029011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:43.029011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:43.029011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:43.029011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:43.029011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:43.029011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:43.029011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:43.029011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:43.029011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:43.029011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:43.029011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:43.029011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:43.029011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:43.029011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:43.029011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:43.030080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:43.030080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:43.030080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:43.030080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:43.030080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:43.030080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:43.030080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:43.030080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:43.030080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:43.030080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:43.030080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:43.030080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:43.030080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:43.030080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:43.030080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:43.030080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:43.030999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:43.030999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:43.030999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:43.030999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:43.030999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:43.030999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:43.030999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:43.030999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:43.030999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:43.030999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:43.030999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:43.030999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:43.030999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 576/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:21:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:43.036000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1716d29-d68f-43d8-8d88-f6b5dc8c51e4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:43.120627 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:43.129723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1716d29-d68f-43d8-8d88-f6b5dc8c51e4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:43.162383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc4d31-7970-467b-9d9d-d450d712ba91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:43.163028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc4d31-7970-467b-9d9d-d450d712ba91 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:43.163309 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc4d31-7970-467b-9d9d-d450d712ba91 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:43.163584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc4d31-7970-467b-9d9d-d450d712ba91 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:43.164314 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc4d31-7970-467b-9d9d-d450d712ba91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:43.164618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc4d31-7970-467b-9d9d-d450d712ba91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:43.166008 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc4d31-7970-467b-9d9d-d450d712ba91 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:43.174038 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 574/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:43.179785 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133ffc5-be12-46de-ad52-70338537b43c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:43.269420 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:43.275781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133ffc5-be12-46de-ad52-70338537b43c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:43.306747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d67f31-80f6-482a-adc4-bcffcec687fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:43.307639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d67f31-80f6-482a-adc4-bcffcec687fe None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:43.307914 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d67f31-80f6-482a-adc4-bcffcec687fe None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:43.308183 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d67f31-80f6-482a-adc4-bcffcec687fe None admin] PATH_INFO: `/v3/users/68909600b94c4291862dc76586ed9a00` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:43.309886 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d67f31-80f6-482a-adc4-bcffcec687fe None admin] RBAC: Authorizing `identity:get_user(user_id=68909600b94c4291862dc76586ed9a00)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:43.311304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d67f31-80f6-482a-adc4-bcffcec687fe None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:43.312916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 577/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:43 2022] GET /identity/v3/users/68909600b94c4291862dc76586ed9a00 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:43.318608 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc4d31-7970-467b-9d9d-d450d712ba91 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:43.417485 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc4d31-7970-467b-9d9d-d450d712ba91 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:43.450113 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382981e-56f5-4fbf-a637-b78983617e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:43.450947 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382981e-56f5-4fbf-a637-b78983617e1e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:43.451221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382981e-56f5-4fbf-a637-b78983617e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:43.451487 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382981e-56f5-4fbf-a637-b78983617e1e None admin] PATH_INFO: `/v3/projects/643922019b474c089b72b74ccfc30675` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:43.452701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382981e-56f5-4fbf-a637-b78983617e1e None admin] RBAC: Authorizing `identity:get_project(project_id=643922019b474c089b72b74ccfc30675)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:43.454136 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382981e-56f5-4fbf-a637-b78983617e1e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:43.455813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 575/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:43 2022] GET /identity/v3/projects/643922019b474c089b72b74ccfc30675 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:43.461450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d67f31-80f6-482a-adc4-bcffcec687fe None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:43.555274 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d67f31-80f6-482a-adc4-bcffcec687fe None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:43.587117 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03549132-9da4-4209-b191-eda47910cc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:43.587983 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03549132-9da4-4209-b191-eda47910cc5f None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:43.588260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03549132-9da4-4209-b191-eda47910cc5f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:43.588531 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03549132-9da4-4209-b191-eda47910cc5f None admin] PATH_INFO: `/v3/projects/643922019b474c089b72b74ccfc30675/users/68909600b94c4291862dc76586ed9a00/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:43.590496 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03549132-9da4-4209-b191-eda47910cc5f None admin] RBAC: Authorizing `identity:create_grant(project_id=643922019b474c089b72b74ccfc30675, user_id=68909600b94c4291862dc76586ed9a00, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:43.591902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03549132-9da4-4209-b191-eda47910cc5f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:43.685775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 578/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:21:43 2022] PUT /identity/v3/projects/643922019b474c089b72b74ccfc30675/users/68909600b94c4291862dc76586ed9a00/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:21:45.875353 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6ba6f-cb97-41e6-b679-96042f13d9ac None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:45.875902 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6ba6f-cb97-41e6-b679-96042f13d9ac None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:45.875902 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6ba6f-cb97-41e6-b679-96042f13d9ac None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:45.877307 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 576/1154] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:45.882495 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a30cc2-6fee-4f0b-b1a7-5b7770b06d0f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:45.882921 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a30cc2-6fee-4f0b-b1a7-5b7770b06d0f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:45.883031 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a30cc2-6fee-4f0b-b1a7-5b7770b06d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:45.899249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e5a30cc2-6fee-4f0b-b1a7-5b7770b06d0f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:45.900406 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e5a30cc2-6fee-4f0b-b1a7-5b7770b06d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:45.916439 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:45.966000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 579/1155] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:45.989888 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6ba6f-cb97-41e6-b679-96042f13d9ac None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:46.011895 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:46.087367 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6ba6f-cb97-41e6-b679-96042f13d9ac None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:46.119618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c0c6d-0ca1-48c5-a9be-341ac4b81642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:46.120392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c0c6d-0ca1-48c5-a9be-341ac4b81642 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:46.121268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c0c6d-0ca1-48c5-a9be-341ac4b81642 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:46.121555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c0c6d-0ca1-48c5-a9be-341ac4b81642 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:46.131359 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c0c6d-0ca1-48c5-a9be-341ac4b81642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:21:46.138876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e32c0c6d-0ca1-48c5-a9be-341ac4b81642 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:46.138876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:46.138876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:46.138876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:46.138876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:46.138876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:46.138876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:46.138876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:46.138876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:46.138876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:46.138876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:46.138876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:46.138876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:21:46.138876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:21:46.138876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:21:46.138876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:21:46.139954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:21:46.139954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:21:46.139954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:21:46.139954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:21:46.139954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:46.139954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:46.139954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:46.139954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:46.139954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:46.139954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:46.139954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:46.139954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:46.139954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:46.139954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:46.139954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:46.139954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:46.140961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:46.140961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:46.140961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:46.140961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:46.140961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:21:46.140961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:21:46.140961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:21:46.140961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:21:46.140961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:21:46.140961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:21:46.140961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:21:46.140961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:21:46.140961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 577/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:21:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:46.145821 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a30cc2-6fee-4f0b-b1a7-5b7770b06d0f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:46.245225 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a30cc2-6fee-4f0b-b1a7-5b7770b06d0f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:46.278485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfbeb61-06a6-4655-8cbf-cefc6171f6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:46.279134 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbeb61-06a6-4655-8cbf-cefc6171f6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:46.279410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbeb61-06a6-4655-8cbf-cefc6171f6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:46.279682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbeb61-06a6-4655-8cbf-cefc6171f6c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:46.280415 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfbeb61-06a6-4655-8cbf-cefc6171f6c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:46.280717 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfbeb61-06a6-4655-8cbf-cefc6171f6c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:46.282105 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfbeb61-06a6-4655-8cbf-cefc6171f6c2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:46.289851 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 580/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:21:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:46.296162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c0c6d-0ca1-48c5-a9be-341ac4b81642 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:46.374099 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:46.392190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c0c6d-0ca1-48c5-a9be-341ac4b81642 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:46.425876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81d776-020d-4901-9df2-4272f0ac4c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:46.426732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81d776-020d-4901-9df2-4272f0ac4c74 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:46.427009 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81d776-020d-4901-9df2-4272f0ac4c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:46.427280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81d776-020d-4901-9df2-4272f0ac4c74 None admin] PATH_INFO: `/v3/users/68909600b94c4291862dc76586ed9a00` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:46.428429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81d776-020d-4901-9df2-4272f0ac4c74 None admin] RBAC: Authorizing `identity:get_user(user_id=68909600b94c4291862dc76586ed9a00)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:46.429881 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81d776-020d-4901-9df2-4272f0ac4c74 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:46.431458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 578/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:21:46 2022] GET /identity/v3/users/68909600b94c4291862dc76586ed9a00 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:46.437085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfbeb61-06a6-4655-8cbf-cefc6171f6c2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:46.453195 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:46.532574 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfbeb61-06a6-4655-8cbf-cefc6171f6c2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:46.564405 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa4281-cb55-4ab5-8bcb-6cd05fabd2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:46.565048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa4281-cb55-4ab5-8bcb-6cd05fabd2cd None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:46.565329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa4281-cb55-4ab5-8bcb-6cd05fabd2cd None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:46.565598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa4281-cb55-4ab5-8bcb-6cd05fabd2cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:46.628448 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa4281-cb55-4ab5-8bcb-6cd05fabd2cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:46.629907 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa4281-cb55-4ab5-8bcb-6cd05fabd2cd None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:46.643806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:46.723264 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 581/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:21:46.729132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81d776-020d-4901-9df2-4272f0ac4c74 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:46.823435 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81d776-020d-4901-9df2-4272f0ac4c74 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:46.981450 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e7bd7-eaea-4892-84a7-3b4f760650a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:46.982298 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e7bd7-eaea-4892-84a7-3b4f760650a0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:46.982581 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e7bd7-eaea-4892-84a7-3b4f760650a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:46.982854 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e7bd7-eaea-4892-84a7-3b4f760650a0 None admin] PATH_INFO: `/v3/projects/643922019b474c089b72b74ccfc30675` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:46.984019 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e7bd7-eaea-4892-84a7-3b4f760650a0 None admin] RBAC: Authorizing `identity:get_project(project_id=643922019b474c089b72b74ccfc30675)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:46.985448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e7bd7-eaea-4892-84a7-3b4f760650a0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:46.987111 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 579/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:21:46 2022] GET /identity/v3/projects/643922019b474c089b72b74ccfc30675 => generated 311 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:46.993247 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa4281-cb55-4ab5-8bcb-6cd05fabd2cd None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:47.088290 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa4281-cb55-4ab5-8bcb-6cd05fabd2cd None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:47.102026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:47.121646 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8516a-faca-413c-86de-9a8e786dade2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:47.125186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8516a-faca-413c-86de-9a8e786dade2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:47.125186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8516a-faca-413c-86de-9a8e786dade2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:47.125186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8516a-faca-413c-86de-9a8e786dade2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:47.125186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8516a-faca-413c-86de-9a8e786dade2 None admin] RBAC: Adding query filter params (role.id=8c63c00418a346b29f7db3f87a404e70, scope.project.id=643922019b474c089b72b74ccfc30675, user.id=68909600b94c4291862dc76586ed9a00) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:47.125186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8516a-faca-413c-86de-9a8e786dade2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:47.125584 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8516a-faca-413c-86de-9a8e786dade2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:47.130910 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 582/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:21:46 2022] GET /identity/v3/role_assignments?user.id=68909600b94c4291862dc76586ed9a00&scope.project.id=643922019b474c089b72b74ccfc30675&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:49.351361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f62eca-bc65-4ee4-a77e-56f96988dc5b None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:49.351903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f62eca-bc65-4ee4-a77e-56f96988dc5b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:49.351903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f62eca-bc65-4ee4-a77e-56f96988dc5b None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:49.353280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 580/1162] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:49.358505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693909d4-0ed9-4376-a0c3-da84b81f09c4 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:49.358925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693909d4-0ed9-4376-a0c3-da84b81f09c4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:49.359047 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693909d4-0ed9-4376-a0c3-da84b81f09c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:49.375682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-693909d4-0ed9-4376-a0c3-da84b81f09c4 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:49.376833 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-693909d4-0ed9-4376-a0c3-da84b81f09c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:49.440866 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 583/1163] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:49.463275 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f62eca-bc65-4ee4-a77e-56f96988dc5b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:49.480690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:49.558067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f62eca-bc65-4ee4-a77e-56f96988dc5b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:49.590067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d04205-9e59-40e9-9419-ed47e0778654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:49.590682 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d04205-9e59-40e9-9419-ed47e0778654 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:49.590959 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d04205-9e59-40e9-9419-ed47e0778654 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:49.591229 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d04205-9e59-40e9-9419-ed47e0778654 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:49.653646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d04205-9e59-40e9-9419-ed47e0778654 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:49.655055 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d04205-9e59-40e9-9419-ed47e0778654 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:49.670892 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:49.747804 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 581/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:21:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:21:49.753573 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693909d4-0ed9-4376-a0c3-da84b81f09c4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:49.846895 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693909d4-0ed9-4376-a0c3-da84b81f09c4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:49.856252 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:49.880345 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b86532-c2ae-4cb3-af7f-257e4399c21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:49.881385 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b86532-c2ae-4cb3-af7f-257e4399c21a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:49.881702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b86532-c2ae-4cb3-af7f-257e4399c21a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:49.882004 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b86532-c2ae-4cb3-af7f-257e4399c21a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:49.888603 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b86532-c2ae-4cb3-af7f-257e4399c21a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:49.890058 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b86532-c2ae-4cb3-af7f-257e4399c21a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:49.903564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-06b86532-c2ae-4cb3-af7f-257e4399c21a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 07 08:21:49.903564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:49.903564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 07 08:21:49.903564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 07 08:21:49.903564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:21:49.903564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:21:49.903564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:21:49.903564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:21:49.903564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 07 08:21:49.903564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:49.903564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:21:49.903564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:49.903564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:21:49.903564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:21:49.903564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:21:49.903564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:21:49.903564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:21:49.904899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:21:49.904899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:21:49.904899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:21:49.904899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:21:49.904899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:21:49.904899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:21:49.904899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:21:49.904899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:21:49.904899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:21:49.904899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:21:49.904899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:21:49.904899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:21:49.904899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:21:49.904899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:21:49.904899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:21:49.904899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:21:49.906186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:21:49.906186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:21:49.906186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:21:49.906186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:21:49.906186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:21:49.906186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:21:49.906186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:21:49.906186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:21:49.906186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:21:49.906186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:21:49.906186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 07 08:21:49.906186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:21:49.906186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 07 08:21:49.906186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:21:49.906186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 584/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jun 7 08:21:49 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:21:49.910723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d04205-9e59-40e9-9419-ed47e0778654 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:50.008228 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d04205-9e59-40e9-9419-ed47e0778654 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:50.024462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:50.042033 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37da47d-1287-4233-9a1a-e6b25df39169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:50.043476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37da47d-1287-4233-9a1a-e6b25df39169 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:50.043755 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37da47d-1287-4233-9a1a-e6b25df39169 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:50.044025 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37da47d-1287-4233-9a1a-e6b25df39169 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:50.044793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37da47d-1287-4233-9a1a-e6b25df39169 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:21:50.045128 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37da47d-1287-4233-9a1a-e6b25df39169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:50.046477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37da47d-1287-4233-9a1a-e6b25df39169 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:50.054766 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 582/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Jun 7 08:21:49 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:52.271820 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d99f2-6c94-41ea-b30c-c8f55493b527 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:52.272313 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d99f2-6c94-41ea-b30c-c8f55493b527 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:52.272405 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d99f2-6c94-41ea-b30c-c8f55493b527 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:52.273764 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 585/1167] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:52.279484 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff89f7-73f0-4b1a-91d4-ae6ac333d91f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:52.279906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff89f7-73f0-4b1a-91d4-ae6ac333d91f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:52.280017 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff89f7-73f0-4b1a-91d4-ae6ac333d91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:52.296583 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7bff89f7-73f0-4b1a-91d4-ae6ac333d91f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:52.298041 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7bff89f7-73f0-4b1a-91d4-ae6ac333d91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:52.360428 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 583/1168] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:52.382540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d99f2-6c94-41ea-b30c-c8f55493b527 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:52.482572 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d99f2-6c94-41ea-b30c-c8f55493b527 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:52.514017 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6992a5-1be1-4418-a395-2b18550fa75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:52.514652 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6992a5-1be1-4418-a395-2b18550fa75a None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:52.514935 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6992a5-1be1-4418-a395-2b18550fa75a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:52.515211 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6992a5-1be1-4418-a395-2b18550fa75a None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:52.515951 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6992a5-1be1-4418-a395-2b18550fa75a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:52.517362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6992a5-1be1-4418-a395-2b18550fa75a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:52.595484 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 586/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Tue Jun 7 08:21:52 2022] POST /identity/v3/domains => generated 267 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:21:54.837820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97083c1-8e91-4721-874b-a7864ddc8b4c None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:54.838345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97083c1-8e91-4721-874b-a7864ddc8b4c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:54.838435 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97083c1-8e91-4721-874b-a7864ddc8b4c None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:54.839728 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 584/1170] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:54.844905 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fc2ca-7661-4ba9-b33c-2d5f2f24ca84 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:54.845316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fc2ca-7661-4ba9-b33c-2d5f2f24ca84 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:54.845502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fc2ca-7661-4ba9-b33c-2d5f2f24ca84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:54.862438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1f8fc2ca-7661-4ba9-b33c-2d5f2f24ca84 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:54.863620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1f8fc2ca-7661-4ba9-b33c-2d5f2f24ca84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:54.886021 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:54.929111 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 587/1171] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:54.953346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97083c1-8e91-4721-874b-a7864ddc8b4c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:55.048942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97083c1-8e91-4721-874b-a7864ddc8b4c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:55.058919 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:55.081443 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bf99c-1bb5-42f1-8723-6a8cada887f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:55.082280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bf99c-1bb5-42f1-8723-6a8cada887f8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:55.082553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bf99c-1bb5-42f1-8723-6a8cada887f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:55.082815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bf99c-1bb5-42f1-8723-6a8cada887f8 None admin] PATH_INFO: `/v3/domains/063dc854d3324afe8f435fcb202bbf03` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:55.093340 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bf99c-1bb5-42f1-8723-6a8cada887f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=063dc854d3324afe8f435fcb202bbf03)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:55.094700 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bf99c-1bb5-42f1-8723-6a8cada887f8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:55.096272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 585/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Tue Jun 7 08:21:54 2022] GET /identity/v3/domains/063dc854d3324afe8f435fcb202bbf03 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:55.101881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fc2ca-7661-4ba9-b33c-2d5f2f24ca84 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:55.195932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fc2ca-7661-4ba9-b33c-2d5f2f24ca84 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:55.229090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd2cf9-bb11-4dad-8425-ea42b197cc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:55.229737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd2cf9-bb11-4dad-8425-ea42b197cc86 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:55.230045 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd2cf9-bb11-4dad-8425-ea42b197cc86 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:55.230321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd2cf9-bb11-4dad-8425-ea42b197cc86 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:55.231022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd2cf9-bb11-4dad-8425-ea42b197cc86 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:55.232458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd2cf9-bb11-4dad-8425-ea42b197cc86 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:55.320673 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 588/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jun 7 08:21:55 2022] POST /identity/v3/projects => generated 361 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:21:57.561846 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac6cad-055b-49d4-87f6-2fc685b24847 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:57.562344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac6cad-055b-49d4-87f6-2fc685b24847 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:57.562344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac6cad-055b-49d4-87f6-2fc685b24847 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:57.563676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 586/1174] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:21:57.568822 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6285f7a-8f1e-4fbc-a8cb-8a1ac275af06 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:57.569219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6285f7a-8f1e-4fbc-a8cb-8a1ac275af06 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:57.569421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6285f7a-8f1e-4fbc-a8cb-8a1ac275af06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:57.586370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a6285f7a-8f1e-4fbc-a8cb-8a1ac275af06 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:21:57.587547 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a6285f7a-8f1e-4fbc-a8cb-8a1ac275af06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:21:57.590287 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:57.652119 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 589/1175] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:21:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:21:57.676928 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac6cad-055b-49d4-87f6-2fc685b24847 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:57.773885 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac6cad-055b-49d4-87f6-2fc685b24847 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:57.805387 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d65849-2e1b-4818-a09f-8bf414ef036f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:57.806229 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d65849-2e1b-4818-a09f-8bf414ef036f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:57.806508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d65849-2e1b-4818-a09f-8bf414ef036f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:57.806779 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d65849-2e1b-4818-a09f-8bf414ef036f None admin] PATH_INFO: `/v3/domains/063dc854d3324afe8f435fcb202bbf03` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:57.808431 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d65849-2e1b-4818-a09f-8bf414ef036f None admin] RBAC: Authorizing `identity:get_domain(domain_id=063dc854d3324afe8f435fcb202bbf03)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:57.809876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d65849-2e1b-4818-a09f-8bf414ef036f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:57.811432 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 587/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Tue Jun 7 08:21:57 2022] GET /identity/v3/domains/063dc854d3324afe8f435fcb202bbf03 => generated 267 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:21:57.816920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6285f7a-8f1e-4fbc-a8cb-8a1ac275af06 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:21:57.912055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6285f7a-8f1e-4fbc-a8cb-8a1ac275af06 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:21:57.945513 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252487a-2474-4bd8-bb02-92a459baa0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:21:57.946175 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252487a-2474-4bd8-bb02-92a459baa0e6 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:21:57.946460 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252487a-2474-4bd8-bb02-92a459baa0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:21:57.946737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252487a-2474-4bd8-bb02-92a459baa0e6 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:21:57.947439 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252487a-2474-4bd8-bb02-92a459baa0e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:21:57.948936 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252487a-2474-4bd8-bb02-92a459baa0e6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:21:57.958743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:21:58.022437 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 590/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jun 7 08:21:57 2022] POST /identity/v3/users => generated 311 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:22:00.280329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd76d8b-785a-450b-9bab-c3cebd1ce353 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:00.280819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd76d8b-785a-450b-9bab-c3cebd1ce353 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:00.280932 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd76d8b-785a-450b-9bab-c3cebd1ce353 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:00.282248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 588/1178] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:00.287458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cfdd0-df3e-4a39-a894-019615b83c4d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:00.287866 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cfdd0-df3e-4a39-a894-019615b83c4d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:00.288570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cfdd0-df3e-4a39-a894-019615b83c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:00.305172 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-836cfdd0-df3e-4a39-a894-019615b83c4d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:00.306330 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-836cfdd0-df3e-4a39-a894-019615b83c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:00.368690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 591/1179] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:22:00.392137 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd76d8b-785a-450b-9bab-c3cebd1ce353 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:00.431694 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:00.489755 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd76d8b-785a-450b-9bab-c3cebd1ce353 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:00.521317 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52c568-e0af-4dc4-a347-f0218880d90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:00.522091 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52c568-e0af-4dc4-a347-f0218880d90b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:00.522372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52c568-e0af-4dc4-a347-f0218880d90b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:00.522645 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52c568-e0af-4dc4-a347-f0218880d90b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:00.532754 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52c568-e0af-4dc4-a347-f0218880d90b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:00.540212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-3a52c568-e0af-4dc4-a347-f0218880d90b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:22:00.540212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:00.540212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:00.540212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:00.540212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:00.540212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:00.540212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:00.540212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:00.540212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:00.540212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:00.540212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:00.540212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:00.540212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:00.540212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:00.540212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:00.540212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:00.541341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:00.541341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:00.541341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:00.541341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:00.541341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:00.541341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:00.541341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:00.541341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:00.541341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:00.541341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:00.541341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:00.541341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:00.541341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:00.541341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:00.541341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:00.541341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:00.542977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:00.542977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:00.542977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:00.542977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:00.542977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:00.542977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:00.542977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:00.542977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:00.542977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:00.542977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:00.542977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:22:00.542977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:22:00.542977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 589/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:22:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:00.547989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836cfdd0-df3e-4a39-a894-019615b83c4d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:00.641002 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836cfdd0-df3e-4a39-a894-019615b83c4d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:00.665844 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:00.674837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72170fe0-4a41-4adb-8b32-eeb285dc0ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:00.675602 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72170fe0-4a41-4adb-8b32-eeb285dc0ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:00.675881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72170fe0-4a41-4adb-8b32-eeb285dc0ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:00.676150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72170fe0-4a41-4adb-8b32-eeb285dc0ac7 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:00.676886 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72170fe0-4a41-4adb-8b32-eeb285dc0ac7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:00.677338 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72170fe0-4a41-4adb-8b32-eeb285dc0ac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:00.678779 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72170fe0-4a41-4adb-8b32-eeb285dc0ac7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:00.686608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 592/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:22:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:00.692059 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52c568-e0af-4dc4-a347-f0218880d90b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:00.783553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:00.787101 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52c568-e0af-4dc4-a347-f0218880d90b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:00.820329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409bec9a-a7dd-4893-a0f5-b9f1203416e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:00.821199 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bec9a-a7dd-4893-a0f5-b9f1203416e8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:00.821477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bec9a-a7dd-4893-a0f5-b9f1203416e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:00.821747 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bec9a-a7dd-4893-a0f5-b9f1203416e8 None admin] PATH_INFO: `/v3/users/c3f2bdc9a02f4df7875954f25c7f5975` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:00.834116 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409bec9a-a7dd-4893-a0f5-b9f1203416e8 None admin] RBAC: Authorizing `identity:get_user(user_id=c3f2bdc9a02f4df7875954f25c7f5975)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:00.835541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409bec9a-a7dd-4893-a0f5-b9f1203416e8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:00.837170 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 590/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:22:00 2022] GET /identity/v3/users/c3f2bdc9a02f4df7875954f25c7f5975 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:00.843347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72170fe0-4a41-4adb-8b32-eeb285dc0ac7 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:00.938217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72170fe0-4a41-4adb-8b32-eeb285dc0ac7 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:00.970697 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e73bd8-7b12-43de-8e07-9d71dda602fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:00.971326 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e73bd8-7b12-43de-8e07-9d71dda602fa None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:00.971607 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e73bd8-7b12-43de-8e07-9d71dda602fa None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:00.971882 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e73bd8-7b12-43de-8e07-9d71dda602fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:01.023746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:01.047003 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e73bd8-7b12-43de-8e07-9d71dda602fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:01.047003 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e73bd8-7b12-43de-8e07-9d71dda602fa None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:01.130326 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 593/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:22:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:22:01.136673 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409bec9a-a7dd-4893-a0f5-b9f1203416e8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:01.232535 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409bec9a-a7dd-4893-a0f5-b9f1203416e8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:01.265535 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670e26f-76f5-40e7-a3ea-17155bf16fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:01.266386 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670e26f-76f5-40e7-a3ea-17155bf16fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:01.266663 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670e26f-76f5-40e7-a3ea-17155bf16fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:01.266933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670e26f-76f5-40e7-a3ea-17155bf16fd0 None admin] PATH_INFO: `/v3/projects/b20ea7609af44aa29c6c5b677870b64d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:01.268082 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670e26f-76f5-40e7-a3ea-17155bf16fd0 None admin] RBAC: Authorizing `identity:get_project(project_id=b20ea7609af44aa29c6c5b677870b64d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:01.269532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670e26f-76f5-40e7-a3ea-17155bf16fd0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:01.271202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 591/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:22:01 2022] GET /identity/v3/projects/b20ea7609af44aa29c6c5b677870b64d => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:01.277202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e73bd8-7b12-43de-8e07-9d71dda602fa None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:01.359259 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:01.371007 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e73bd8-7b12-43de-8e07-9d71dda602fa None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:01.404023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53203a3c-adeb-4add-bbd7-e3297613e87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:01.404687 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53203a3c-adeb-4add-bbd7-e3297613e87f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:01.404967 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53203a3c-adeb-4add-bbd7-e3297613e87f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:01.405275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53203a3c-adeb-4add-bbd7-e3297613e87f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:01.406103 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53203a3c-adeb-4add-bbd7-e3297613e87f None admin] RBAC: Adding query filter params (role.id=8c63c00418a346b29f7db3f87a404e70, scope.project.id=b20ea7609af44aa29c6c5b677870b64d, user.id=c3f2bdc9a02f4df7875954f25c7f5975) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:01.406417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53203a3c-adeb-4add-bbd7-e3297613e87f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:01.407835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53203a3c-adeb-4add-bbd7-e3297613e87f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:01.413185 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 594/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:22:01 2022] GET /identity/v3/role_assignments?user.id=c3f2bdc9a02f4df7875954f25c7f5975&scope.project.id=b20ea7609af44aa29c6c5b677870b64d&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:03.593652 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6ef3d-635c-4fb4-ad19-390b99594dd0 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:03.594152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6ef3d-635c-4fb4-ad19-390b99594dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:03.594152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6ef3d-635c-4fb4-ad19-390b99594dd0 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:03.595473 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 592/1186] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:03.600640 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956177d7-6c9a-43a1-b6a9-103e2a17cb9b None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:03.601102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956177d7-6c9a-43a1-b6a9-103e2a17cb9b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:03.601230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956177d7-6c9a-43a1-b6a9-103e2a17cb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:03.617520 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-956177d7-6c9a-43a1-b6a9-103e2a17cb9b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:03.618694 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-956177d7-6c9a-43a1-b6a9-103e2a17cb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:03.683519 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 595/1187] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:22:03.705836 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6ef3d-635c-4fb4-ad19-390b99594dd0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:03.729714 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:03.801955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6ef3d-635c-4fb4-ad19-390b99594dd0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:03.834382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b4564-bd3d-4045-b146-9c9c3425f658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:03.835151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b4564-bd3d-4045-b146-9c9c3425f658 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:03.835433 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b4564-bd3d-4045-b146-9c9c3425f658 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:03.835705 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b4564-bd3d-4045-b146-9c9c3425f658 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:03.845438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b4564-bd3d-4045-b146-9c9c3425f658 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:03.852847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9a3b4564-bd3d-4045-b146-9c9c3425f658 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:22:03.852847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:03.852847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:03.852847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:03.852847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:03.852847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:03.852847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:03.852847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:03.852847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:03.852847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:03.852847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:03.852847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:03.852847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:03.852847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:03.852847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:03.852847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:03.853973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:03.853973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:03.853973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:03.853973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:03.853973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:03.853973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:03.853973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:03.853973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:03.853973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:03.853973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:03.853973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:03.853973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:03.853973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:03.853973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:03.853973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:03.853973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:03.855452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:03.855452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:03.855452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:03.855452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:03.855452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:03.855452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:03.855452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:03.855452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:03.855452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:03.855452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:03.855452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:22:03.855452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:22:03.855452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 593/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:22:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:03.860257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956177d7-6c9a-43a1-b6a9-103e2a17cb9b None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:03.938988 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:03.956189 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956177d7-6c9a-43a1-b6a9-103e2a17cb9b None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:03.989415 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f92819-663f-4de2-92f3-10f40d34b0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:03.990076 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f92819-663f-4de2-92f3-10f40d34b0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:03.990363 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f92819-663f-4de2-92f3-10f40d34b0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:03.990642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f92819-663f-4de2-92f3-10f40d34b0f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:03.991390 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f92819-663f-4de2-92f3-10f40d34b0f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:03.991698 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f92819-663f-4de2-92f3-10f40d34b0f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:03.993089 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f92819-663f-4de2-92f3-10f40d34b0f6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:04.001660 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 596/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:22:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:04.007419 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b4564-bd3d-4045-b146-9c9c3425f658 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:04.088055 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:04.103873 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b4564-bd3d-4045-b146-9c9c3425f658 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:04.137434 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a00391-8c84-41b8-9dda-cc46c58fb349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:04.138290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a00391-8c84-41b8-9dda-cc46c58fb349 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:04.138568 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a00391-8c84-41b8-9dda-cc46c58fb349 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:04.138840 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a00391-8c84-41b8-9dda-cc46c58fb349 None admin] PATH_INFO: `/v3/users/c3f2bdc9a02f4df7875954f25c7f5975` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:04.139990 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a00391-8c84-41b8-9dda-cc46c58fb349 None admin] RBAC: Authorizing `identity:get_user(user_id=c3f2bdc9a02f4df7875954f25c7f5975)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:04.141443 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a00391-8c84-41b8-9dda-cc46c58fb349 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:04.143030 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 594/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:22:04 2022] GET /identity/v3/users/c3f2bdc9a02f4df7875954f25c7f5975 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:04.148545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f92819-663f-4de2-92f3-10f40d34b0f6 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:04.248371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f92819-663f-4de2-92f3-10f40d34b0f6 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:04.280331 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9155a8f-ecd8-422e-be40-dc391432e0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:04.281203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9155a8f-ecd8-422e-be40-dc391432e0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:04.281484 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9155a8f-ecd8-422e-be40-dc391432e0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:04.281758 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9155a8f-ecd8-422e-be40-dc391432e0a6 None admin] PATH_INFO: `/v3/projects/b20ea7609af44aa29c6c5b677870b64d` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:04.282914 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9155a8f-ecd8-422e-be40-dc391432e0a6 None admin] RBAC: Authorizing `identity:get_project(project_id=b20ea7609af44aa29c6c5b677870b64d)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:04.284343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9155a8f-ecd8-422e-be40-dc391432e0a6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:04.286021 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 597/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:22:04 2022] GET /identity/v3/projects/b20ea7609af44aa29c6c5b677870b64d => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:04.291814 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a00391-8c84-41b8-9dda-cc46c58fb349 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:04.386976 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a00391-8c84-41b8-9dda-cc46c58fb349 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:04.418718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c6968-0f33-46b6-b53a-a60b587d0e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:04.419588 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c6968-0f33-46b6-b53a-a60b587d0e33 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:04.419866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c6968-0f33-46b6-b53a-a60b587d0e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:04.420137 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c6968-0f33-46b6-b53a-a60b587d0e33 None admin] PATH_INFO: `/v3/projects/b20ea7609af44aa29c6c5b677870b64d/users/c3f2bdc9a02f4df7875954f25c7f5975/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:04.422083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c6968-0f33-46b6-b53a-a60b587d0e33 None admin] RBAC: Authorizing `identity:create_grant(project_id=b20ea7609af44aa29c6c5b677870b64d, user_id=c3f2bdc9a02f4df7875954f25c7f5975, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:04.423751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c6968-0f33-46b6-b53a-a60b587d0e33 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:04.507244 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 595/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:22:04 2022] PUT /identity/v3/projects/b20ea7609af44aa29c6c5b677870b64d/users/c3f2bdc9a02f4df7875954f25c7f5975/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:22:06.685201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247549b-88de-4185-9a2e-f7d6763f785b None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:06.685748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247549b-88de-4185-9a2e-f7d6763f785b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:06.685748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247549b-88de-4185-9a2e-f7d6763f785b None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:06.687114 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 598/1193] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:22:06.692358 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c11319-a51c-4a48-95e2-44cae143878d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:06.692735 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c11319-a51c-4a48-95e2-44cae143878d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:06.692913 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c11319-a51c-4a48-95e2-44cae143878d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:06.709935 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-45c11319-a51c-4a48-95e2-44cae143878d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:22:06.711105 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-45c11319-a51c-4a48-95e2-44cae143878d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:22:06.718659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:06.775678 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 596/1194] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:22:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:22:06.800077 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247549b-88de-4185-9a2e-f7d6763f785b None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:06.814945 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:06.897795 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247549b-88de-4185-9a2e-f7d6763f785b None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:06.930097 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db032ef8-c462-4c47-983a-2644e7e8fba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:06.930868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db032ef8-c462-4c47-983a-2644e7e8fba1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:06.931146 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db032ef8-c462-4c47-983a-2644e7e8fba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:06.931414 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db032ef8-c462-4c47-983a-2644e7e8fba1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:06.941607 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db032ef8-c462-4c47-983a-2644e7e8fba1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:22:06.949469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-db032ef8-c462-4c47-983a-2644e7e8fba1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:22:06.949469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:22:06.949469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:22:06.949469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:22:06.949469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:22:06.949469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:22:06.949469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:22:06.949469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:22:06.949469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:22:06.949469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:22:06.949469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:22:06.949469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:22:06.949469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:22:06.949469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:22:06.949469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:22:06.949469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:22:06.950563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:22:06.950563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:22:06.950563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:22:06.950563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:22:06.950563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:22:06.950563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:22:06.950563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:22:06.950563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:22:06.950563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:22:06.950563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:22:06.950563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:22:06.950563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:22:06.950563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:22:06.950563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:22:06.950563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:22:06.950563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:22:06.951521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:22:06.951521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:22:06.951521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:22:06.951521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:22:06.951521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:22:06.951521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:22:06.951521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:22:06.951521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:22:06.951521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:22:06.951521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:22:06.951521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:22:06.951521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:22:06.951521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 599/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:22:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:22:06.956586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c11319-a51c-4a48-95e2-44cae143878d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:07.049884 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c11319-a51c-4a48-95e2-44cae143878d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:07.082109 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce48660-bfc7-44ad-a964-f40fcd8c450c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:07.082754 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce48660-bfc7-44ad-a964-f40fcd8c450c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:07.083032 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce48660-bfc7-44ad-a964-f40fcd8c450c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:07.083307 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce48660-bfc7-44ad-a964-f40fcd8c450c None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:07.084034 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce48660-bfc7-44ad-a964-f40fcd8c450c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:07.084336 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce48660-bfc7-44ad-a964-f40fcd8c450c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:07.085704 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce48660-bfc7-44ad-a964-f40fcd8c450c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:07.093931 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 597/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:22:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:07.099316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db032ef8-c462-4c47-983a-2644e7e8fba1 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:07.165284 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:07.193122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db032ef8-c462-4c47-983a-2644e7e8fba1 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:07.227127 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2762738-7589-4d50-a7c7-40efb890a598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:07.227983 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2762738-7589-4d50-a7c7-40efb890a598 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:07.228265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2762738-7589-4d50-a7c7-40efb890a598 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:07.228541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2762738-7589-4d50-a7c7-40efb890a598 None admin] PATH_INFO: `/v3/users/c3f2bdc9a02f4df7875954f25c7f5975` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:07.229728 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2762738-7589-4d50-a7c7-40efb890a598 None admin] RBAC: Authorizing `identity:get_user(user_id=c3f2bdc9a02f4df7875954f25c7f5975)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:07.231159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2762738-7589-4d50-a7c7-40efb890a598 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:07.232759 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 600/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Jun 7 08:22:07 2022] GET /identity/v3/users/c3f2bdc9a02f4df7875954f25c7f5975 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:07.238386 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce48660-bfc7-44ad-a964-f40fcd8c450c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:07.242091 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:07.334033 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce48660-bfc7-44ad-a964-f40fcd8c450c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:07.366227 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949cb869-a39d-4277-9882-acf0a127ad84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:07.366857 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949cb869-a39d-4277-9882-acf0a127ad84 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:07.367137 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949cb869-a39d-4277-9882-acf0a127ad84 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:07.367410 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949cb869-a39d-4277-9882-acf0a127ad84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:07.430880 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949cb869-a39d-4277-9882-acf0a127ad84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:07.432299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949cb869-a39d-4277-9882-acf0a127ad84 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:07.435111 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:07.525764 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 598/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:22:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:22:07.532003 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2762738-7589-4d50-a7c7-40efb890a598 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:07.626734 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2762738-7589-4d50-a7c7-40efb890a598 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:07.659153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9dc073-c8d4-4798-9e34-f4f28bd63423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:07.659980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dc073-c8d4-4798-9e34-f4f28bd63423 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:07.660257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dc073-c8d4-4798-9e34-f4f28bd63423 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:07.660524 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dc073-c8d4-4798-9e34-f4f28bd63423 None admin] PATH_INFO: `/v3/projects/b20ea7609af44aa29c6c5b677870b64d` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:07.661698 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9dc073-c8d4-4798-9e34-f4f28bd63423 None admin] RBAC: Authorizing `identity:get_project(project_id=b20ea7609af44aa29c6c5b677870b64d)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:07.663106 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9dc073-c8d4-4798-9e34-f4f28bd63423 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:07.664707 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 601/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Jun 7 08:22:07 2022] GET /identity/v3/projects/b20ea7609af44aa29c6c5b677870b64d => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:07.670651 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949cb869-a39d-4277-9882-acf0a127ad84 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:22:07.763776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949cb869-a39d-4277-9882-acf0a127ad84 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:22:07.765983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:07.795668 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0933eea-9e67-428a-aee2-4389dee03155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:22:07.796247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0933eea-9e67-428a-aee2-4389dee03155 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:22:07.796525 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0933eea-9e67-428a-aee2-4389dee03155 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:22:07.796798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0933eea-9e67-428a-aee2-4389dee03155 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:22:07.797903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0933eea-9e67-428a-aee2-4389dee03155 None admin] RBAC: Adding query filter params (role.id=8c63c00418a346b29f7db3f87a404e70, scope.project.id=b20ea7609af44aa29c6c5b677870b64d, user.id=c3f2bdc9a02f4df7875954f25c7f5975) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:22:07.798250 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0933eea-9e67-428a-aee2-4389dee03155 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:22:07.799648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0933eea-9e67-428a-aee2-4389dee03155 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:22:07.805137 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 599/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:22:07 2022] GET /identity/v3/role_assignments?user.id=c3f2bdc9a02f4df7875954f25c7f5975&scope.project.id=b20ea7609af44aa29c6c5b677870b64d&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:22:17.655337 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=96 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:22:17.816715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=16 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:22.806956 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889be1c0-91c4-466d-9648-071ac1d51100 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:22.807549 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889be1c0-91c4-466d-9648-071ac1d51100 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:22.807549 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889be1c0-91c4-466d-9648-071ac1d51100 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:22.808892 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 602/1201] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:22.814251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831a006-c44a-4f88-b795-dd22e13e2573 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:22.814699 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831a006-c44a-4f88-b795-dd22e13e2573 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:22.814833 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831a006-c44a-4f88-b795-dd22e13e2573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:22.831933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5831a006-c44a-4f88-b795-dd22e13e2573 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:22.833135 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5831a006-c44a-4f88-b795-dd22e13e2573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:22.896751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 600/1202] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 08:23:22.922389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889be1c0-91c4-466d-9648-071ac1d51100 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:23.024659 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889be1c0-91c4-466d-9648-071ac1d51100 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:23.057573 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3aab58-a109-4364-a813-4f094062d709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:23.058817 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3aab58-a109-4364-a813-4f094062d709 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:23.059099 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3aab58-a109-4364-a813-4f094062d709 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:23.059377 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3aab58-a109-4364-a813-4f094062d709 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:23.069166 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3aab58-a109-4364-a813-4f094062d709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:23.070616 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3aab58-a109-4364-a813-4f094062d709 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:23.087971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-bb3aab58-a109-4364-a813-4f094062d709 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:23.087971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:23.087971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:23.087971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:23.087971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:23.087971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:23.087971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:23.087971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:23.087971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:23.087971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:23.087971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:23.087971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:23.087971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:23.087971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:23.087971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:23.089266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:23.089266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:23.089266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:23.089266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:23.089266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:23.089266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:23.089266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:23.089266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:23.089266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:23.089266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:23.089266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:23.089266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:23.089266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:23.089266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:23.089266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:23.089266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:23.089266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:23.090729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:23.090729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:23.090729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:23.090729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:23:23.090729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 603/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:23.095239 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831a006-c44a-4f88-b795-dd22e13e2573 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:23.191872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831a006-c44a-4f88-b795-dd22e13e2573 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:23.220041 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:23.225584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db371372-630f-4642-9859-60670a04086a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:23.226241 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db371372-630f-4642-9859-60670a04086a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:23.226521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db371372-630f-4642-9859-60670a04086a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:23.226791 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db371372-630f-4642-9859-60670a04086a None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:23.227532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db371372-630f-4642-9859-60670a04086a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:23.227834 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db371372-630f-4642-9859-60670a04086a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:23.229209 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db371372-630f-4642-9859-60670a04086a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:23.240023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 601/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:23.246267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3aab58-a109-4364-a813-4f094062d709 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:23.299190 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:23.343313 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3aab58-a109-4364-a813-4f094062d709 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:23.376650 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d78e5-be7e-44db-8f59-9f24f1a07518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:23.386788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d78e5-be7e-44db-8f59-9f24f1a07518 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:23.386788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d78e5-be7e-44db-8f59-9f24f1a07518 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:23.386788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d78e5-be7e-44db-8f59-9f24f1a07518 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:23.386788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d78e5-be7e-44db-8f59-9f24f1a07518 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:23.386788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d78e5-be7e-44db-8f59-9f24f1a07518 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:23.471963 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 604/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Jun 7 08:23:23 2022] POST /identity/v3/users => generated 251 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:23:25.726704 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c700064-75a1-4df0-b637-01962eba5a31 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:25.727272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c700064-75a1-4df0-b637-01962eba5a31 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:25.727272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c700064-75a1-4df0-b637-01962eba5a31 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:25.728691 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 602/1206] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:25.734057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f219f-e57d-41a4-a583-68808b9d38ce None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:25.734480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f219f-e57d-41a4-a583-68808b9d38ce None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:25.734628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f219f-e57d-41a4-a583-68808b9d38ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:25.750900 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ad1f219f-e57d-41a4-a583-68808b9d38ce None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:25.752068 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ad1f219f-e57d-41a4-a583-68808b9d38ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:25.817424 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 605/1207] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:25.842233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c700064-75a1-4df0-b637-01962eba5a31 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:25.941510 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c700064-75a1-4df0-b637-01962eba5a31 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:25.974441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96542eb-d71f-42dc-a276-36502c64cba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:25.975256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96542eb-d71f-42dc-a276-36502c64cba9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:25.975536 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96542eb-d71f-42dc-a276-36502c64cba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:25.975806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96542eb-d71f-42dc-a276-36502c64cba9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:25.990706 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96542eb-d71f-42dc-a276-36502c64cba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:23:25.993575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d96542eb-d71f-42dc-a276-36502c64cba9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:23:25.993575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:25.993575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:25.993575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:25.993575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:25.993575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:25.993575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:25.993575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:25.993575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:25.993575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:25.993575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:25.993575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:25.993575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:23:25.993575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:23:25.993575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:23:25.993575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:23:25.994532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:23:25.994532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:23:25.994532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:23:25.994532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:23:25.994532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:25.994532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:25.994532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:25.994532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:25.994532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:25.994532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:25.994532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:25.994532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:25.994532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:25.994532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:25.994532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:25.994532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:25.996110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:25.996110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:25.996110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:25.996110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:25.996110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:23:25.996110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:23:25.996110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:23:25.996110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:23:25.996110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:23:25.996110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:23:25.996110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:23:25.996110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:25.996110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 603/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:23:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:26.001261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f219f-e57d-41a4-a583-68808b9d38ce None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:26.040338 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:26.097947 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f219f-e57d-41a4-a583-68808b9d38ce None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:26.130140 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3899e7c-330f-4e1e-beac-2e1037864c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:26.130794 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3899e7c-330f-4e1e-beac-2e1037864c04 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:26.131079 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3899e7c-330f-4e1e-beac-2e1037864c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:26.131354 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3899e7c-330f-4e1e-beac-2e1037864c04 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:26.132093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3899e7c-330f-4e1e-beac-2e1037864c04 None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:26.132404 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3899e7c-330f-4e1e-beac-2e1037864c04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:26.133800 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3899e7c-330f-4e1e-beac-2e1037864c04 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:26.142100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 606/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:23:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:26.147781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96542eb-d71f-42dc-a276-36502c64cba9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:26.199197 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:26.246127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96542eb-d71f-42dc-a276-36502c64cba9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:26.278240 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63176179-dc95-47ae-aebd-1facbdbbc0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:26.279023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63176179-dc95-47ae-aebd-1facbdbbc0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:26.279304 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63176179-dc95-47ae-aebd-1facbdbbc0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:26.279579 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63176179-dc95-47ae-aebd-1facbdbbc0f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:26.289711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63176179-dc95-47ae-aebd-1facbdbbc0f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:26.291142 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63176179-dc95-47ae-aebd-1facbdbbc0f9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:26.308782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-63176179-dc95-47ae-aebd-1facbdbbc0f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:26.308782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:26.308782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:26.308782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:26.308782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:26.308782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:26.308782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:26.308782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:26.308782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:26.308782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:26.308782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:26.308782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:26.308782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:26.308782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:26.308782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:26.309903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:26.309903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:26.309903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:26.309903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:26.309903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:26.309903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:26.309903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:26.309903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:26.309903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:26.309903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:26.309903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:26.309903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:26.309903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:26.309903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:26.309903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:26.309903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:26.309903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:26.311166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:26.311166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:26.311166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:26.311166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:26.311166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 604/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:26.315751 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3899e7c-330f-4e1e-beac-2e1037864c04 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:26.412336 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:26.415430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3899e7c-330f-4e1e-beac-2e1037864c04 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:26.448629 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af20a8b-ffd0-4930-9e92-c0cebf1ada72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:26.449313 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af20a8b-ffd0-4930-9e92-c0cebf1ada72 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:26.449596 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af20a8b-ffd0-4930-9e92-c0cebf1ada72 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:26.449871 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af20a8b-ffd0-4930-9e92-c0cebf1ada72 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:26.450584 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af20a8b-ffd0-4930-9e92-c0cebf1ada72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:26.450888 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af20a8b-ffd0-4930-9e92-c0cebf1ada72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:26.452253 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af20a8b-ffd0-4930-9e92-c0cebf1ada72 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:26.463477 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 607/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:26.469162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63176179-dc95-47ae-aebd-1facbdbbc0f9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:26.567019 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63176179-dc95-47ae-aebd-1facbdbbc0f9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:26.568445 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:26.599117 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0f849-8eb1-4fa3-8ccc-b85c81436aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:26.599912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0f849-8eb1-4fa3-8ccc-b85c81436aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:26.600194 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0f849-8eb1-4fa3-8ccc-b85c81436aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:26.600467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0f849-8eb1-4fa3-8ccc-b85c81436aa3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:26.607687 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0f849-8eb1-4fa3-8ccc-b85c81436aa3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:26.609856 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0f849-8eb1-4fa3-8ccc-b85c81436aa3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:26.624001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-0bf0f849-8eb1-4fa3-8ccc-b85c81436aa3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:23:26.624001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:26.624001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:26.624001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:26.624001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:26.624001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:26.624001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:26.624001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:26.624001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:26.624001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:26.624001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:26.624001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:26.624001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:23:26.624001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:23:26.624001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:23:26.624001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:23:26.625129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:26.625129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:26.625129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:23:26.625129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:26.625129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:23:26.625129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:26.625129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:26.625129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:26.625129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:26.625129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:26.625129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:26.625129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:26.625129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:26.625129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:26.625129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:26.625129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:26.625129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:26.625129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:26.626465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:26.626465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:26.626465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:23:26.626465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:23:26.626465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:23:26.626465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:23:26.626465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:23:26.626465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:23:26.626465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:23:26.626465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:26.626465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 605/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:26.631130 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af20a8b-ffd0-4930-9e92-c0cebf1ada72 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:26.728004 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af20a8b-ffd0-4930-9e92-c0cebf1ada72 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:26.762670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b59acd-dec1-4813-ab24-3e4380f60cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:26.763324 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b59acd-dec1-4813-ab24-3e4380f60cbc None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:26.763599 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b59acd-dec1-4813-ab24-3e4380f60cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:26.763866 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b59acd-dec1-4813-ab24-3e4380f60cbc None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:26.764683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b59acd-dec1-4813-ab24-3e4380f60cbc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:26.764984 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b59acd-dec1-4813-ab24-3e4380f60cbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:26.766459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b59acd-dec1-4813-ab24-3e4380f60cbc None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:26.784135 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 608/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Jun 7 08:23:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:26.789825 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0f849-8eb1-4fa3-8ccc-b85c81436aa3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:26.888147 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0f849-8eb1-4fa3-8ccc-b85c81436aa3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:26.921692 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd7dee-edf7-4878-94d9-03f3539c003e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:26.922432 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd7dee-edf7-4878-94d9-03f3539c003e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:26.922709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd7dee-edf7-4878-94d9-03f3539c003e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:26.922980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd7dee-edf7-4878-94d9-03f3539c003e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:26.950721 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:26.988004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd7dee-edf7-4878-94d9-03f3539c003e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:26.989457 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd7dee-edf7-4878-94d9-03f3539c003e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:27.082124 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 606/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:23:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:23:27.088069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b59acd-dec1-4813-ab24-3e4380f60cbc None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:27.107130 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:27.184663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b59acd-dec1-4813-ab24-3e4380f60cbc None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:27.218590 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74751eec-3581-475d-9aee-09eecd7f3031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:27.219373 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74751eec-3581-475d-9aee-09eecd7f3031 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:27.219657 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74751eec-3581-475d-9aee-09eecd7f3031 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:27.219933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74751eec-3581-475d-9aee-09eecd7f3031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:27.229539 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74751eec-3581-475d-9aee-09eecd7f3031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:27.231022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74751eec-3581-475d-9aee-09eecd7f3031 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:27.247747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-74751eec-3581-475d-9aee-09eecd7f3031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:27.247747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:27.247747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:27.247747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:27.247747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:27.247747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:27.247747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:27.247747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:27.247747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:27.247747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:27.247747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:27.247747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:27.247747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:27.247747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:27.247747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:27.248690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:27.248690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:27.248690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:27.248690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:27.248690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:27.248690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:27.248690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:27.248690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:27.248690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:27.248690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:27.248690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:27.248690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:27.248690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:27.248690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:27.248690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:27.248690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:27.248690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:27.249703 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:27.249703 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:27.249703 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:27.249703 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:23:27.249703 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 609/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:27.255095 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd7dee-edf7-4878-94d9-03f3539c003e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:27.314512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:27.352719 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd7dee-edf7-4878-94d9-03f3539c003e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:27.385275 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db251f01-10dc-44c5-818d-ff7b255598a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:27.385923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db251f01-10dc-44c5-818d-ff7b255598a5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:27.386724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db251f01-10dc-44c5-818d-ff7b255598a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:27.387036 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db251f01-10dc-44c5-818d-ff7b255598a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:27.387741 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db251f01-10dc-44c5-818d-ff7b255598a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:27.388044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db251f01-10dc-44c5-818d-ff7b255598a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:27.389417 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db251f01-10dc-44c5-818d-ff7b255598a5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:27.401136 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 607/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:27.407412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74751eec-3581-475d-9aee-09eecd7f3031 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:27.476439 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:27.506422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74751eec-3581-475d-9aee-09eecd7f3031 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:27.538983 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fd912-7e16-4cc5-8150-1a843655d18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:27.539796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fd912-7e16-4cc5-8150-1a843655d18c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:27.540078 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fd912-7e16-4cc5-8150-1a843655d18c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:27.540355 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fd912-7e16-4cc5-8150-1a843655d18c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:27.547016 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fd912-7e16-4cc5-8150-1a843655d18c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:27.548492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fd912-7e16-4cc5-8150-1a843655d18c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:27.563005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-162fd912-7e16-4cc5-8150-1a843655d18c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:27.563005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:27.563005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:27.563005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:27.563005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:27.563005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:27.563005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:27.563005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:27.563005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:27.563005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:27.563005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:27.563005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:27.563005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:23:27.563005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:23:27.563005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:23:27.564073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:23:27.564073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:27.564073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:27.564073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:27.564073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:27.564073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:27.564073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:27.564073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:27.564073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:27.564073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:27.564073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:27.564073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:27.564073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:27.564073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:27.564073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:27.564073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:27.564073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:27.565215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:23:27.565215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:23:27.565215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:23:27.565215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:23:27.565215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:23:27.565215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:23:27.565215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:27.565215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:23:27.565215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 610/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:23:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:27.570079 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db251f01-10dc-44c5-818d-ff7b255598a5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:27.668515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db251f01-10dc-44c5-818d-ff7b255598a5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:27.690144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:27.701287 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc545f57-b1c4-4c1b-9bfe-0d4bd9c00b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:27.701936 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc545f57-b1c4-4c1b-9bfe-0d4bd9c00b06 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:27.702213 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc545f57-b1c4-4c1b-9bfe-0d4bd9c00b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:27.702482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc545f57-b1c4-4c1b-9bfe-0d4bd9c00b06 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:27.703249 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc545f57-b1c4-4c1b-9bfe-0d4bd9c00b06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:27.703550 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc545f57-b1c4-4c1b-9bfe-0d4bd9c00b06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:27.704953 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc545f57-b1c4-4c1b-9bfe-0d4bd9c00b06 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:27.716648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 608/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:23:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:27.722687 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fd912-7e16-4cc5-8150-1a843655d18c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:27.821082 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fd912-7e16-4cc5-8150-1a843655d18c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:27.849748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:27.855566 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68d8ad-4506-44e1-889a-56a749eb59ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:27.856208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68d8ad-4506-44e1-889a-56a749eb59ee None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:27.856494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68d8ad-4506-44e1-889a-56a749eb59ee None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:27.856771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68d8ad-4506-44e1-889a-56a749eb59ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:27.857620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68d8ad-4506-44e1-889a-56a749eb59ee None admin] RBAC: Adding query filter params (role.id=a04f2ab5f1234bd6b1dc4c18b4210a89, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=1fc0d7bd7e4547f4a8d90ffc2c2e1d03) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:27.857933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68d8ad-4506-44e1-889a-56a749eb59ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:27.859356 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68d8ad-4506-44e1-889a-56a749eb59ee None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:27.864727 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 611/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:23:27 2022] GET /identity/v3/role_assignments?user.id=1fc0d7bd7e4547f4a8d90ffc2c2e1d03&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:30.054701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f8ac4-78a9-48a7-aa75-a9ab0a726bd5 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:30.055271 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f8ac4-78a9-48a7-aa75-a9ab0a726bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:30.055271 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f8ac4-78a9-48a7-aa75-a9ab0a726bd5 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:30.056610 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 609/1220] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:30.061759 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d5084-033e-4fe3-b42a-b550c1c1dd59 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:30.062178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d5084-033e-4fe3-b42a-b550c1c1dd59 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:30.062324 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d5084-033e-4fe3-b42a-b550c1c1dd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:30.079286 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-2c7d5084-033e-4fe3-b42a-b550c1c1dd59 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:30.080467 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2c7d5084-033e-4fe3-b42a-b550c1c1dd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:30.145640 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 612/1221] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:30.168106 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f8ac4-78a9-48a7-aa75-a9ab0a726bd5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:30.272546 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f8ac4-78a9-48a7-aa75-a9ab0a726bd5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:30.308284 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c43da-ef7a-4afe-8ac6-9dada4c28d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:30.309036 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c43da-ef7a-4afe-8ac6-9dada4c28d30 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:30.309314 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c43da-ef7a-4afe-8ac6-9dada4c28d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:30.309583 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c43da-ef7a-4afe-8ac6-9dada4c28d30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:30.319317 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096c43da-ef7a-4afe-8ac6-9dada4c28d30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:23:30.327463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-096c43da-ef7a-4afe-8ac6-9dada4c28d30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:23:30.327463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:30.327463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:30.327463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:30.327463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:30.327463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:30.327463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:30.327463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:30.327463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:30.327463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:30.327463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:30.327463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:30.327463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:23:30.327463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:23:30.327463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:23:30.327463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:23:30.328512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:23:30.328512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:23:30.328512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:23:30.328512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:23:30.328512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:30.328512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:30.328512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:30.328512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:30.328512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:30.328512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:30.328512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:30.328512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:30.328512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:30.328512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:30.328512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:30.328512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:30.329634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:30.329634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:30.329634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:30.329634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:30.329634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:23:30.329634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:23:30.329634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:23:30.329634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:23:30.329634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:23:30.329634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:23:30.329634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:23:30.329634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:30.329634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 610/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:23:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:30.334651 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d5084-033e-4fe3-b42a-b550c1c1dd59 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:30.431446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d5084-033e-4fe3-b42a-b550c1c1dd59 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:30.454731 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:30.465027 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5b9bb-51a4-4c9e-8a72-fea0e9ee2fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:30.465693 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5b9bb-51a4-4c9e-8a72-fea0e9ee2fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:30.465973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5b9bb-51a4-4c9e-8a72-fea0e9ee2fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:30.466243 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5b9bb-51a4-4c9e-8a72-fea0e9ee2fe3 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:30.466976 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5b9bb-51a4-4c9e-8a72-fea0e9ee2fe3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:30.467280 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5b9bb-51a4-4c9e-8a72-fea0e9ee2fe3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:30.468642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5b9bb-51a4-4c9e-8a72-fea0e9ee2fe3 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:30.476172 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 613/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:23:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:30.482046 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c43da-ef7a-4afe-8ac6-9dada4c28d30 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:30.524576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:30.580580 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c43da-ef7a-4afe-8ac6-9dada4c28d30 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:30.612391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d3915-183e-4483-b091-518327d82593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:30.613196 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d3915-183e-4483-b091-518327d82593 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:30.613472 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d3915-183e-4483-b091-518327d82593 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:30.613740 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d3915-183e-4483-b091-518327d82593 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:30.624605 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d3915-183e-4483-b091-518327d82593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:30.626058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d3915-183e-4483-b091-518327d82593 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:30.643018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-677d3915-183e-4483-b091-518327d82593 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:30.643018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:30.643018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:30.643018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:30.643018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:30.643018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:30.643018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:30.643018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:30.643018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:30.643018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:30.643018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:30.643018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:30.643018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:30.643018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:30.643018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:30.644100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:30.644100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:30.644100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:30.644100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:30.644100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:30.644100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:30.644100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:30.644100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:30.644100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:30.644100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:30.644100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:30.644100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:30.644100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:30.644100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:30.644100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:30.644100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:30.644100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:30.645206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:30.645206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:30.645206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:30.645206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:30.645206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 611/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:30.650120 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5b9bb-51a4-4c9e-8a72-fea0e9ee2fe3 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:30.747594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5b9bb-51a4-4c9e-8a72-fea0e9ee2fe3 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:30.780203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188a54e-f304-44ab-acd1-f2d97a498082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:30.780846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188a54e-f304-44ab-acd1-f2d97a498082 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:30.781162 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188a54e-f304-44ab-acd1-f2d97a498082 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:30.781434 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188a54e-f304-44ab-acd1-f2d97a498082 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:30.782144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b188a54e-f304-44ab-acd1-f2d97a498082 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:30.782446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b188a54e-f304-44ab-acd1-f2d97a498082 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:30.783804 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b188a54e-f304-44ab-acd1-f2d97a498082 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:30.795675 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 614/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:30.801592 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d3915-183e-4483-b091-518327d82593 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:30.892066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:30.898458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d3915-183e-4483-b091-518327d82593 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:30.931139 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b532f-148a-4eee-b73e-c139c466ef9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:30.931929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b532f-148a-4eee-b73e-c139c466ef9e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:30.932210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b532f-148a-4eee-b73e-c139c466ef9e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:30.932485 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b532f-148a-4eee-b73e-c139c466ef9e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:30.939871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175b532f-148a-4eee-b73e-c139c466ef9e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:30.941330 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175b532f-148a-4eee-b73e-c139c466ef9e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:30.955566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-175b532f-148a-4eee-b73e-c139c466ef9e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:23:30.955566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:30.955566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:30.955566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:30.955566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:30.955566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:30.955566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:30.955566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:30.955566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:30.955566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:30.955566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:30.955566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:30.955566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:23:30.955566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:23:30.955566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:23:30.955566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:23:30.956655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:30.956655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:30.956655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:23:30.956655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:30.956655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:23:30.956655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:30.956655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:30.956655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:30.956655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:30.956655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:30.956655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:30.956655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:30.956655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:30.956655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:30.956655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:30.956655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:30.956655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:30.956655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:30.957843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:30.957843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:30.957843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:23:30.957843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:23:30.957843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:23:30.957843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:23:30.957843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:23:30.957843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:23:30.957843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:23:30.957843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:30.957843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 612/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:30.962699 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188a54e-f304-44ab-acd1-f2d97a498082 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:31.010763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:31.059802 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188a54e-f304-44ab-acd1-f2d97a498082 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:31.092617 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d154ad-e155-47b0-a2ff-d0062822427d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:31.093379 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d154ad-e155-47b0-a2ff-d0062822427d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:31.093587 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d154ad-e155-47b0-a2ff-d0062822427d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:31.093859 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d154ad-e155-47b0-a2ff-d0062822427d None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:31.094692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d154ad-e155-47b0-a2ff-d0062822427d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:31.094999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d154ad-e155-47b0-a2ff-d0062822427d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:31.096435 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d154ad-e155-47b0-a2ff-d0062822427d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:31.114225 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 615/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Jun 7 08:23:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:31.119949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b532f-148a-4eee-b73e-c139c466ef9e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:31.218405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b532f-148a-4eee-b73e-c139c466ef9e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:31.251413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53aba25-7d77-4b2a-956b-053926cbaaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:31.252281 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53aba25-7d77-4b2a-956b-053926cbaaff None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:31.252672 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53aba25-7d77-4b2a-956b-053926cbaaff None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:31.252945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53aba25-7d77-4b2a-956b-053926cbaaff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:31.262777 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53aba25-7d77-4b2a-956b-053926cbaaff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:31.264256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53aba25-7d77-4b2a-956b-053926cbaaff None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:31.281752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-f53aba25-7d77-4b2a-956b-053926cbaaff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:31.281752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:31.281752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:31.281752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:31.281752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:31.281752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:31.281752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:31.281752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:31.281752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:31.281752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:31.281752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:31.281752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:31.281752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:31.281752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:31.281752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:31.282802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:31.282802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:31.282802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:31.282802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:31.282802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:31.282802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:31.282802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:31.282802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:31.282802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:31.282802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:31.282802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:31.282802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:31.282802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:31.282802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:31.282802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:31.282802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:31.282802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:31.283875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:31.283875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:31.283875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:31.283875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:31.283875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 613/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:31.288785 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d154ad-e155-47b0-a2ff-d0062822427d None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:31.388007 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d154ad-e155-47b0-a2ff-d0062822427d None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:31.390325 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:31.421763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e2bbb-89a8-4746-8000-228b040213e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:31.422421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e2bbb-89a8-4746-8000-228b040213e3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:31.422704 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e2bbb-89a8-4746-8000-228b040213e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:31.422982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e2bbb-89a8-4746-8000-228b040213e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:31.423700 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e2bbb-89a8-4746-8000-228b040213e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:31.424008 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e2bbb-89a8-4746-8000-228b040213e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:31.425437 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e2bbb-89a8-4746-8000-228b040213e3 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:31.437058 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 616/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:31.442625 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53aba25-7d77-4b2a-956b-053926cbaaff None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:31.451318 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:31.540183 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53aba25-7d77-4b2a-956b-053926cbaaff None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:31.572661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29922c69-a837-4327-a5ef-1b08bbca8f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:31.574012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29922c69-a837-4327-a5ef-1b08bbca8f16 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:31.574301 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29922c69-a837-4327-a5ef-1b08bbca8f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:31.574569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29922c69-a837-4327-a5ef-1b08bbca8f16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:31.581111 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29922c69-a837-4327-a5ef-1b08bbca8f16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:31.582549 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29922c69-a837-4327-a5ef-1b08bbca8f16 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:31.597016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-29922c69-a837-4327-a5ef-1b08bbca8f16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:31.597016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:31.597016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:31.597016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:31.597016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:31.597016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:31.597016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:31.597016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:31.597016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:31.597016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:31.597016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:31.597016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:31.597016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:23:31.597016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:23:31.597016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:23:31.598023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:23:31.598023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:31.598023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:31.598023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:31.598023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:31.598023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:31.598023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:31.598023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:31.598023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:31.598023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:31.598023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:31.598023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:31.598023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:31.598023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:31.598023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:31.598023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:31.598023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:31.599371 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:23:31.599371 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:23:31.599371 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:23:31.599371 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:23:31.599371 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:23:31.599371 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:23:31.599371 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:31.599371 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:31.599371 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 614/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:23:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:31.603929 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e2bbb-89a8-4746-8000-228b040213e3 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:31.701262 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e2bbb-89a8-4746-8000-228b040213e3 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:31.733817 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667193a3-c64b-45d7-876d-fe5f20cb070a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:31.734521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667193a3-c64b-45d7-876d-fe5f20cb070a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:31.734725 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667193a3-c64b-45d7-876d-fe5f20cb070a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:31.734999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667193a3-c64b-45d7-876d-fe5f20cb070a None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:31.735776 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667193a3-c64b-45d7-876d-fe5f20cb070a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:31.736083 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667193a3-c64b-45d7-876d-fe5f20cb070a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:31.737541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667193a3-c64b-45d7-876d-fe5f20cb070a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:31.749022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 617/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:23:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:31.755005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29922c69-a837-4327-a5ef-1b08bbca8f16 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:31.818982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:31.852876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29922c69-a837-4327-a5ef-1b08bbca8f16 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:31.885504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a54f1-83fe-4bde-8f80-8ce65cb3da6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:31.886387 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a54f1-83fe-4bde-8f80-8ce65cb3da6b None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:31.886664 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a54f1-83fe-4bde-8f80-8ce65cb3da6b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:31.886934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a54f1-83fe-4bde-8f80-8ce65cb3da6b None admin] PATH_INFO: `/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/1fc0d7bd7e4547f4a8d90ffc2c2e1d03/roles/a04f2ab5f1234bd6b1dc4c18b4210a89` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:31.901973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a54f1-83fe-4bde-8f80-8ce65cb3da6b None admin] RBAC: Authorizing `identity:create_grant(project_id=afc8781e5f864d4a9374300ee9b5193b, user_id=1fc0d7bd7e4547f4a8d90ffc2c2e1d03, role_id=a04f2ab5f1234bd6b1dc4c18b4210a89)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:31.903550 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a54f1-83fe-4bde-8f80-8ce65cb3da6b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:31.957565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 615/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:23:31 2022] PUT /identity/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/1fc0d7bd7e4547f4a8d90ffc2c2e1d03/roles/a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:23:34.155382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4977a46-0e2b-4d89-99fd-96393396aefb None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:34.156109 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4977a46-0e2b-4d89-99fd-96393396aefb None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:34.156109 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4977a46-0e2b-4d89-99fd-96393396aefb None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:34.157320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 618/1233] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:34.162418 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e55510-f78d-446a-8635-dbcb3c04bd53 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:34.162825 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e55510-f78d-446a-8635-dbcb3c04bd53 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:34.162968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e55510-f78d-446a-8635-dbcb3c04bd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:34.182115 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b0e55510-f78d-446a-8635-dbcb3c04bd53 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:34.183293 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b0e55510-f78d-446a-8635-dbcb3c04bd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:34.248270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 616/1234] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:34.272011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4977a46-0e2b-4d89-99fd-96393396aefb None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:34.302508 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:34.379838 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4977a46-0e2b-4d89-99fd-96393396aefb None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:34.411257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb882be9-deef-46fa-8823-d4286a5671f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:34.411976 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb882be9-deef-46fa-8823-d4286a5671f5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:34.412260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb882be9-deef-46fa-8823-d4286a5671f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:34.412530 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb882be9-deef-46fa-8823-d4286a5671f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:34.422503 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb882be9-deef-46fa-8823-d4286a5671f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:23:34.430036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-bb882be9-deef-46fa-8823-d4286a5671f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:23:34.430036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:34.430036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:34.430036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:34.430036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:34.430036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:34.430036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:34.430036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:34.430036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:34.430036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:34.430036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:34.430036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:34.430036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:23:34.430036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:23:34.430036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:23:34.430036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:23:34.431138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:23:34.431138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:23:34.431138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:23:34.431138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:23:34.431138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:34.431138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:34.431138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:34.431138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:34.431138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:34.431138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:34.431138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:34.431138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:34.431138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:34.431138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:34.431138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:34.431138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:34.432102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:34.432102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:34.432102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:34.432102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:34.432102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:23:34.432102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:23:34.432102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:23:34.432102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:23:34.432102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:23:34.432102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:23:34.432102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 08:23:34.432102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:23:34.432102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 619/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Jun 7 08:23:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:34.437765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e55510-f78d-446a-8635-dbcb3c04bd53 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:34.481641 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:34.532208 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e55510-f78d-446a-8635-dbcb3c04bd53 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:34.566319 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21873c27-b979-4308-b697-9657b8e53a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:34.566975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21873c27-b979-4308-b697-9657b8e53a8e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:34.567258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21873c27-b979-4308-b697-9657b8e53a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:34.567576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21873c27-b979-4308-b697-9657b8e53a8e None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:34.568311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21873c27-b979-4308-b697-9657b8e53a8e None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:34.568614 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21873c27-b979-4308-b697-9657b8e53a8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:34.570005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21873c27-b979-4308-b697-9657b8e53a8e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:34.577582 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 617/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Jun 7 08:23:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:34.583240 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb882be9-deef-46fa-8823-d4286a5671f5 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:34.671339 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:34.687001 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb882be9-deef-46fa-8823-d4286a5671f5 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:34.718392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70977f6c-9e51-44dc-a173-60cc48ad6d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:34.719175 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70977f6c-9e51-44dc-a173-60cc48ad6d41 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:34.719461 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70977f6c-9e51-44dc-a173-60cc48ad6d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:34.719739 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70977f6c-9e51-44dc-a173-60cc48ad6d41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:34.730005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70977f6c-9e51-44dc-a173-60cc48ad6d41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:34.731962 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70977f6c-9e51-44dc-a173-60cc48ad6d41 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:34.748980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-70977f6c-9e51-44dc-a173-60cc48ad6d41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:34.748980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:34.748980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:34.748980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:34.748980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:34.748980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:34.748980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:34.748980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:34.748980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:34.748980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:34.748980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:34.748980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:34.748980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:34.748980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:34.748980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:34.751771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:34.751771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:34.751771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:34.751771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:34.751771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:34.751771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:34.751771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:34.751771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:34.751771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:34.751771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:34.751771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:34.751771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:34.751771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:34.751771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:34.751771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:34.751771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:34.751771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:34.752867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:34.752867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:34.752867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:34.752867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:23:34.752867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 620/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:34.755935 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21873c27-b979-4308-b697-9657b8e53a8e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:34.852849 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21873c27-b979-4308-b697-9657b8e53a8e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:34.862293 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:34.886088 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a1d66-0606-4acb-89db-37aa00364f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:34.886744 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1d66-0606-4acb-89db-37aa00364f80 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:34.887025 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1d66-0606-4acb-89db-37aa00364f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:34.887338 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1d66-0606-4acb-89db-37aa00364f80 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:34.888114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a1d66-0606-4acb-89db-37aa00364f80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:34.888396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a1d66-0606-4acb-89db-37aa00364f80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:34.889792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a1d66-0606-4acb-89db-37aa00364f80 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:34.901758 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 618/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:34.907942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70977f6c-9e51-44dc-a173-60cc48ad6d41 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:35.004533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70977f6c-9e51-44dc-a173-60cc48ad6d41 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:35.037140 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ceec0a-5772-4e5f-86fc-5aff73a98886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:35.037928 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ceec0a-5772-4e5f-86fc-5aff73a98886 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:35.038208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ceec0a-5772-4e5f-86fc-5aff73a98886 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:35.038482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ceec0a-5772-4e5f-86fc-5aff73a98886 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:35.041136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:35.046760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ceec0a-5772-4e5f-86fc-5aff73a98886 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:35.048212 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ceec0a-5772-4e5f-86fc-5aff73a98886 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:35.062167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-c3ceec0a-5772-4e5f-86fc-5aff73a98886 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:23:35.062167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:35.062167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:35.062167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:35.062167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:35.062167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:35.062167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:35.062167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:35.062167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:35.062167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:35.062167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:35.062167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:35.062167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:23:35.062167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:23:35.062167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:23:35.062167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:23:35.063341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:35.063341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:35.063341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:23:35.063341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:35.063341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:23:35.063341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:35.063341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:35.063341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:35.063341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:35.063341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:35.063341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:35.063341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:35.063341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:35.063341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:35.063341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:35.063341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:35.063341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:35.063341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:35.064504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:35.064504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:35.064504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:23:35.064504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:23:35.064504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:23:35.064504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:23:35.064504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:23:35.064504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:23:35.064504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:23:35.064504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:23:35.064504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 621/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:35.069236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a1d66-0606-4acb-89db-37aa00364f80 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:35.164969 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a1d66-0606-4acb-89db-37aa00364f80 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:35.198046 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9241e-90ec-429c-b725-3b29e5d338bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:35.198714 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9241e-90ec-429c-b725-3b29e5d338bb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:35.198992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9241e-90ec-429c-b725-3b29e5d338bb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:35.199264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9241e-90ec-429c-b725-3b29e5d338bb None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:35.200090 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb9241e-90ec-429c-b725-3b29e5d338bb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:35.200400 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb9241e-90ec-429c-b725-3b29e5d338bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:35.201854 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb9241e-90ec-429c-b725-3b29e5d338bb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:35.219043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 619/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Jun 7 08:23:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:35.224846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ceec0a-5772-4e5f-86fc-5aff73a98886 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:35.321205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ceec0a-5772-4e5f-86fc-5aff73a98886 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:35.354602 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b3389-9ac6-4a7e-9dbd-08a0291b73ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:35.355231 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b3389-9ac6-4a7e-9dbd-08a0291b73ac None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:35.355515 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b3389-9ac6-4a7e-9dbd-08a0291b73ac None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:35.355782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b3389-9ac6-4a7e-9dbd-08a0291b73ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:35.415039 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:35.421621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b3389-9ac6-4a7e-9dbd-08a0291b73ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:35.423063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b3389-9ac6-4a7e-9dbd-08a0291b73ac None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:35.516815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 622/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:23:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:23:35.522575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9241e-90ec-429c-b725-3b29e5d338bb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:35.550220 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:35.620684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9241e-90ec-429c-b725-3b29e5d338bb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:35.655187 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea899c5c-5b71-4e29-82d7-fe639fddb369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:35.655974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea899c5c-5b71-4e29-82d7-fe639fddb369 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:35.656255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea899c5c-5b71-4e29-82d7-fe639fddb369 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:35.656526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea899c5c-5b71-4e29-82d7-fe639fddb369 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:35.666399 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea899c5c-5b71-4e29-82d7-fe639fddb369 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:35.668926 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea899c5c-5b71-4e29-82d7-fe639fddb369 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:35.685983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-ea899c5c-5b71-4e29-82d7-fe639fddb369 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:35.685983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:35.685983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:35.685983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:35.685983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:35.685983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:35.685983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:35.685983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:35.685983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:35.685983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:35.685983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:35.685983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:35.685983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:35.685983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:35.685983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:35.687003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:35.687003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:35.687003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:35.687003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:35.687003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:35.687003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:35.687003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:35.687003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:35.687003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:35.687003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:35.687003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:35.687003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:35.687003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:35.687003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:35.687003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:35.687003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:35.687003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:35.687923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:35.687923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:35.687923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:35.687923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:35.687923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 620/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:35.693115 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b3389-9ac6-4a7e-9dbd-08a0291b73ac None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:35.783285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:35.788872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b3389-9ac6-4a7e-9dbd-08a0291b73ac None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:35.822362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e371a-b171-4ebf-8273-ccb654775ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:35.823009 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e371a-b171-4ebf-8273-ccb654775ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:35.823292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e371a-b171-4ebf-8273-ccb654775ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:35.823567 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e371a-b171-4ebf-8273-ccb654775ea1 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:35.824367 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e371a-b171-4ebf-8273-ccb654775ea1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:35.825233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e371a-b171-4ebf-8273-ccb654775ea1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:35.826602 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e371a-b171-4ebf-8273-ccb654775ea1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:35.837195 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 623/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:35.842897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea899c5c-5b71-4e29-82d7-fe639fddb369 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:35.918753 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:35.939461 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea899c5c-5b71-4e29-82d7-fe639fddb369 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:35.972890 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3e3be-774b-4ddc-be6d-94e04e3f29c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:35.973738 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3e3be-774b-4ddc-be6d-94e04e3f29c6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:35.974022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3e3be-774b-4ddc-be6d-94e04e3f29c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:35.974300 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3e3be-774b-4ddc-be6d-94e04e3f29c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:35.981015 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3e3be-774b-4ddc-be6d-94e04e3f29c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:35.982798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3e3be-774b-4ddc-be6d-94e04e3f29c6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:35.996441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d1b3e3be-774b-4ddc-be6d-94e04e3f29c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:35.996441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:35.996441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:35.996441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:35.996441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:35.996441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:35.996441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:35.996441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:35.996441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:35.996441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:35.996441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:35.996441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:35.996441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:23:35.996441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:23:35.996441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:23:35.997980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:23:35.997980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:35.997980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:35.997980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:35.997980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:35.997980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:35.997980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:35.997980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:35.997980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:35.997980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:35.997980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:35.997980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:35.997980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:35.997980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:35.997980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:35.997980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:35.997980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:35.998898 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:23:35.998898 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:23:35.998898 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:23:35.998898 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:23:35.998898 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:23:35.998898 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:23:35.998898 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:35.998898 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:35.998898 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 621/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:23:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:36.003848 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e371a-b171-4ebf-8273-ccb654775ea1 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:36.102297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e371a-b171-4ebf-8273-ccb654775ea1 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:36.135419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe63e53-66c0-432f-b515-42e546a60062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:36.136060 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe63e53-66c0-432f-b515-42e546a60062 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:36.136337 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe63e53-66c0-432f-b515-42e546a60062 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:36.136603 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe63e53-66c0-432f-b515-42e546a60062 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:36.137424 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe63e53-66c0-432f-b515-42e546a60062 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:36.137728 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe63e53-66c0-432f-b515-42e546a60062 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:36.139135 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe63e53-66c0-432f-b515-42e546a60062 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:36.150424 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 624/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:23:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:36.156556 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3e3be-774b-4ddc-be6d-94e04e3f29c6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:36.255274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3e3be-774b-4ddc-be6d-94e04e3f29c6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:36.286863 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f9311-47ff-4233-a034-6fedfbb7210d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:36.287506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f9311-47ff-4233-a034-6fedfbb7210d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:36.287785 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f9311-47ff-4233-a034-6fedfbb7210d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:36.288057 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f9311-47ff-4233-a034-6fedfbb7210d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:36.288986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f9311-47ff-4233-a034-6fedfbb7210d None admin] RBAC: Adding query filter params (role.id=a04f2ab5f1234bd6b1dc4c18b4210a89, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=1fc0d7bd7e4547f4a8d90ffc2c2e1d03) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:36.289331 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f9311-47ff-4233-a034-6fedfbb7210d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:36.290748 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f9311-47ff-4233-a034-6fedfbb7210d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:36.297375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 622/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:23:36 2022] GET /identity/v3/role_assignments?user.id=1fc0d7bd7e4547f4a8d90ffc2c2e1d03&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=a04f2ab5f1234bd6b1dc4c18b4210a89 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:38.566648 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b01cf-4d40-4101-b4a1-c0aa714fb173 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:38.567167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b01cf-4d40-4101-b4a1-c0aa714fb173 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:38.567167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b01cf-4d40-4101-b4a1-c0aa714fb173 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:38.568480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 625/1247] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:38.573752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fa8cc-1d8c-438e-bb51-ff9f16942404 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:38.574185 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fa8cc-1d8c-438e-bb51-ff9f16942404 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:38.574375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fa8cc-1d8c-438e-bb51-ff9f16942404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:38.579911 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:38.592289 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-342fa8cc-1d8c-438e-bb51-ff9f16942404 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:38.593495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-342fa8cc-1d8c-438e-bb51-ff9f16942404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:38.658860 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 623/1248] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:38.682393 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b01cf-4d40-4101-b4a1-c0aa714fb173 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:38.701704 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:38.781066 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b01cf-4d40-4101-b4a1-c0aa714fb173 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:38.813078 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc19bc-fd80-4333-99c0-86dcf36b48a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:38.813897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc19bc-fd80-4333-99c0-86dcf36b48a8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:38.814174 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc19bc-fd80-4333-99c0-86dcf36b48a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:38.814445 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc19bc-fd80-4333-99c0-86dcf36b48a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:38.824212 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc19bc-fd80-4333-99c0-86dcf36b48a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:23:38.832312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-30cc19bc-fd80-4333-99c0-86dcf36b48a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:23:38.832312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:38.832312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:38.832312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:38.832312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:38.832312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:38.832312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:38.832312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:38.832312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:38.832312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:38.832312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:38.832312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:38.832312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:23:38.832312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:23:38.832312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:23:38.832312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:23:38.833482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:23:38.833482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:23:38.833482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:23:38.833482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:23:38.833482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:38.833482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:38.833482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:38.833482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:38.833482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:38.833482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:38.833482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:38.833482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:38.833482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:38.833482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:38.833482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:38.833482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:38.834416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:38.834416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:38.834416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:38.834416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:38.834416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:23:38.834416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:23:38.834416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:23:38.834416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:23:38.834416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:23:38.834416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:23:38.834416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:23:38.834416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:23:38.834416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 626/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:23:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:38.839398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fa8cc-1d8c-438e-bb51-ff9f16942404 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:38.938527 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fa8cc-1d8c-438e-bb51-ff9f16942404 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:38.966451 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:38.971129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265c7fe-5781-41ca-b594-f1591cfe98e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:38.971789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265c7fe-5781-41ca-b594-f1591cfe98e4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:38.972070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265c7fe-5781-41ca-b594-f1591cfe98e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:38.972345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265c7fe-5781-41ca-b594-f1591cfe98e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:38.973107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265c7fe-5781-41ca-b594-f1591cfe98e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:38.973516 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265c7fe-5781-41ca-b594-f1591cfe98e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:38.974816 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265c7fe-5781-41ca-b594-f1591cfe98e4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:38.982503 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 624/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:23:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:38.988207 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc19bc-fd80-4333-99c0-86dcf36b48a8 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:39.064656 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:39.086124 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc19bc-fd80-4333-99c0-86dcf36b48a8 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:39.118056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0a775-4f0a-412e-a23c-3396066de431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:39.118821 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0a775-4f0a-412e-a23c-3396066de431 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:39.119098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0a775-4f0a-412e-a23c-3396066de431 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:39.119369 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0a775-4f0a-412e-a23c-3396066de431 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:39.130088 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0a775-4f0a-412e-a23c-3396066de431 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:39.131513 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0a775-4f0a-412e-a23c-3396066de431 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:39.148278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-8bf0a775-4f0a-412e-a23c-3396066de431 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:39.148278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:39.148278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:39.148278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:39.148278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:39.148278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:39.148278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:39.148278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:39.148278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:39.148278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:39.148278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:39.148278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:39.148278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:39.148278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:39.148278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:39.151034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:39.151034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:39.151034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:39.151034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:39.151034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:39.151034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:39.151034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:39.151034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:39.151034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:39.151034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:39.151034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:39.151034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:39.151034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:39.151034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:39.151034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:39.151034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:39.151034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:39.152143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:39.152143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:39.152143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:39.152143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:23:39.152143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 627/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:39.155839 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265c7fe-5781-41ca-b594-f1591cfe98e4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:39.253046 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265c7fe-5781-41ca-b594-f1591cfe98e4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:39.286627 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e31927-08e7-4585-86fa-0a140faf4b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:39.287279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e31927-08e7-4585-86fa-0a140faf4b23 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:39.287557 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e31927-08e7-4585-86fa-0a140faf4b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:39.287857 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e31927-08e7-4585-86fa-0a140faf4b23 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:39.288570 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e31927-08e7-4585-86fa-0a140faf4b23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:39.288870 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e31927-08e7-4585-86fa-0a140faf4b23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:39.290259 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e31927-08e7-4585-86fa-0a140faf4b23 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:39.301824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 625/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:39.307576 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0a775-4f0a-412e-a23c-3396066de431 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:39.406989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0a775-4f0a-412e-a23c-3396066de431 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:39.432632 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:39.440670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c2f29-8315-49a5-af98-ffe7e7045ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:39.441486 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c2f29-8315-49a5-af98-ffe7e7045ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:39.441768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c2f29-8315-49a5-af98-ffe7e7045ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:39.442034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c2f29-8315-49a5-af98-ffe7e7045ad8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:39.449417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c2f29-8315-49a5-af98-ffe7e7045ad8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:39.451422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c2f29-8315-49a5-af98-ffe7e7045ad8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:39.465422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-af1c2f29-8315-49a5-af98-ffe7e7045ad8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:23:39.465422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:39.465422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:39.465422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:39.465422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:39.465422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:39.465422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:39.465422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:39.465422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:39.465422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:39.465422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:39.465422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:39.465422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:23:39.465422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:23:39.465422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:23:39.465422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:23:39.471898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:39.471898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:39.471898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:23:39.471898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:39.471898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:23:39.471898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:39.471898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:39.471898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:39.471898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:39.471898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:39.471898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:39.471898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:39.471898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:39.471898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:39.471898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:39.471898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:39.471898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:39.471898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:39.473141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:39.473141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:39.473141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:23:39.473141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:23:39.473141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:23:39.473141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:23:39.473141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:23:39.473141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:23:39.473141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:23:39.473141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:23:39.473141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 628/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:39.473882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e31927-08e7-4585-86fa-0a140faf4b23 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:39.527732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:39.570722 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e31927-08e7-4585-86fa-0a140faf4b23 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:39.604082 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc967abe-96f3-45e8-b7ed-e3f9e804581a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:39.604751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc967abe-96f3-45e8-b7ed-e3f9e804581a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:39.607248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc967abe-96f3-45e8-b7ed-e3f9e804581a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:39.607248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc967abe-96f3-45e8-b7ed-e3f9e804581a None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:39.607248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc967abe-96f3-45e8-b7ed-e3f9e804581a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:39.607248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc967abe-96f3-45e8-b7ed-e3f9e804581a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:39.607945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc967abe-96f3-45e8-b7ed-e3f9e804581a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:39.625103 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 626/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Jun 7 08:23:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:39.630760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c2f29-8315-49a5-af98-ffe7e7045ad8 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:39.731833 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c2f29-8315-49a5-af98-ffe7e7045ad8 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:39.765764 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b3e3e-7791-4848-adc3-ddf18455e6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:39.766403 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b3e3e-7791-4848-adc3-ddf18455e6aa None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:39.766685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b3e3e-7791-4848-adc3-ddf18455e6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:39.766959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b3e3e-7791-4848-adc3-ddf18455e6aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:39.821394 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:39.833268 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b3e3e-7791-4848-adc3-ddf18455e6aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:39.834720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b3e3e-7791-4848-adc3-ddf18455e6aa None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:39.931765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 629/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:23:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:23:39.938246 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc967abe-96f3-45e8-b7ed-e3f9e804581a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:40.034721 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc967abe-96f3-45e8-b7ed-e3f9e804581a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:40.048210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:40.068776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a65367-4f1d-44ce-8974-a1a1f6119a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:40.069596 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a65367-4f1d-44ce-8974-a1a1f6119a42 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:40.069882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a65367-4f1d-44ce-8974-a1a1f6119a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:40.070160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a65367-4f1d-44ce-8974-a1a1f6119a42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:40.079926 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a65367-4f1d-44ce-8974-a1a1f6119a42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:40.081396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a65367-4f1d-44ce-8974-a1a1f6119a42 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:40.098480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-56a65367-4f1d-44ce-8974-a1a1f6119a42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:40.098480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:40.098480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:40.098480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:40.098480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:40.098480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:40.098480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:40.098480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:40.098480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:40.098480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:40.098480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:40.098480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:40.098480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:40.098480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:40.098480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:40.099521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:40.099521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:40.099521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:40.099521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:40.099521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:40.099521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:40.099521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:40.099521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:40.099521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:40.099521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:40.099521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:40.099521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:40.099521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:40.099521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:40.099521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:40.099521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:40.099521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:40.100808 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:40.100808 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:40.100808 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:40.100808 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:40.100808 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 627/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:40.105336 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b3e3e-7791-4848-adc3-ddf18455e6aa None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:40.194792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:40.206349 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b3e3e-7791-4848-adc3-ddf18455e6aa None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:40.239905 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7d58f-341c-441d-88e7-8d4a9d155dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:40.240554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7d58f-341c-441d-88e7-8d4a9d155dcc None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:40.240835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7d58f-341c-441d-88e7-8d4a9d155dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:40.241140 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7d58f-341c-441d-88e7-8d4a9d155dcc None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:40.241866 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7d58f-341c-441d-88e7-8d4a9d155dcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:40.242167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7d58f-341c-441d-88e7-8d4a9d155dcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:40.243532 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7d58f-341c-441d-88e7-8d4a9d155dcc None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:40.254535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 630/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:40.260351 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a65367-4f1d-44ce-8974-a1a1f6119a42 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:40.359482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a65367-4f1d-44ce-8974-a1a1f6119a42 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:40.391774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8210854-c8cc-44df-a56f-9157833edf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:40.392555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8210854-c8cc-44df-a56f-9157833edf58 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:40.392836 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8210854-c8cc-44df-a56f-9157833edf58 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:40.393140 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8210854-c8cc-44df-a56f-9157833edf58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:40.399944 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8210854-c8cc-44df-a56f-9157833edf58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:40.401436 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8210854-c8cc-44df-a56f-9157833edf58 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:40.415422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-f8210854-c8cc-44df-a56f-9157833edf58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:40.415422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:40.415422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:40.415422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:40.415422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:40.415422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:40.415422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:40.415422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:40.415422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:40.415422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:40.415422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:40.415422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:40.415422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:23:40.415422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:23:40.415422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:23:40.421370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:23:40.421370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:40.421370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:40.421370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:40.421370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:40.421370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:40.421370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:40.421370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:40.421370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:40.421370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:40.421370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:40.421370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:40.421370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:40.421370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:40.421370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:40.421370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:40.421370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:40.422469 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:23:40.422469 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:23:40.422469 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:23:40.422469 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:23:40.422469 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:23:40.422469 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:23:40.422469 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:40.422469 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:40.422469 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 628/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:23:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:40.423433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7d58f-341c-441d-88e7-8d4a9d155dcc None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:40.519375 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7d58f-341c-441d-88e7-8d4a9d155dcc None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:40.552725 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf755a2-ea30-4f06-9d8c-96992576184e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:40.553404 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf755a2-ea30-4f06-9d8c-96992576184e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:40.553686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf755a2-ea30-4f06-9d8c-96992576184e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:40.553958 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf755a2-ea30-4f06-9d8c-96992576184e None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:40.554763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf755a2-ea30-4f06-9d8c-96992576184e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:40.555069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf755a2-ea30-4f06-9d8c-96992576184e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:40.556500 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf755a2-ea30-4f06-9d8c-96992576184e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:40.568139 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 631/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:23:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:40.574289 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8210854-c8cc-44df-a56f-9157833edf58 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:40.600146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:40.672212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8210854-c8cc-44df-a56f-9157833edf58 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:40.705425 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4faf29b-465b-4c3b-a109-6c066346e808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:40.706062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faf29b-465b-4c3b-a109-6c066346e808 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:40.706340 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faf29b-465b-4c3b-a109-6c066346e808 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:40.706608 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faf29b-465b-4c3b-a109-6c066346e808 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:40.707419 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4faf29b-465b-4c3b-a109-6c066346e808 None admin] RBAC: Adding query filter params (role.id=8c63c00418a346b29f7db3f87a404e70, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=1fc0d7bd7e4547f4a8d90ffc2c2e1d03) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:40.707725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4faf29b-465b-4c3b-a109-6c066346e808 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:40.709154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4faf29b-465b-4c3b-a109-6c066346e808 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:40.716615 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 629/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:23:40 2022] GET /identity/v3/role_assignments?user.id=1fc0d7bd7e4547f4a8d90ffc2c2e1d03&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:42.957658 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff91ec-0e49-4f18-b79f-c26f0e7f2ce9 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:42.958190 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff91ec-0e49-4f18-b79f-c26f0e7f2ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:42.958190 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff91ec-0e49-4f18-b79f-c26f0e7f2ce9 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:42.959562 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 632/1261] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:42.964804 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10658e8e-f1cf-4344-a198-7ed2624505a2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:42.965227 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10658e8e-f1cf-4344-a198-7ed2624505a2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:42.965380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10658e8e-f1cf-4344-a198-7ed2624505a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:42.982749 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-10658e8e-f1cf-4344-a198-7ed2624505a2 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:42.983923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-10658e8e-f1cf-4344-a198-7ed2624505a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:43.047755 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 630/1262] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:43.070507 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff91ec-0e49-4f18-b79f-c26f0e7f2ce9 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:43.081697 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:43.168681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff91ec-0e49-4f18-b79f-c26f0e7f2ce9 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:43.202296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38564461-8124-49a5-a8b3-1da3b82ed0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:43.203132 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38564461-8124-49a5-a8b3-1da3b82ed0db None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:43.203504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38564461-8124-49a5-a8b3-1da3b82ed0db None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:43.203776 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38564461-8124-49a5-a8b3-1da3b82ed0db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:43.213684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38564461-8124-49a5-a8b3-1da3b82ed0db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:23:43.221545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-38564461-8124-49a5-a8b3-1da3b82ed0db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:23:43.221545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:43.221545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:43.221545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:43.221545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:43.221545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:43.221545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:43.221545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:43.221545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:43.221545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:43.221545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:43.221545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:43.221545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:23:43.221545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:23:43.221545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:23:43.221545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:23:43.222509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:23:43.222509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:23:43.222509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:23:43.222509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:23:43.222509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:43.222509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:43.222509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:43.222509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:43.222509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:43.222509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:43.222509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:43.222509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:43.222509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:43.222509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:43.222509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:43.222509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:43.223370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:43.223370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:43.223370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:43.223370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:43.223370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:23:43.223370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:23:43.223370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:23:43.223370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:23:43.223370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:23:43.223370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:23:43.223370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:23:43.223370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:23:43.223370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 633/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:23:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:43.228807 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10658e8e-f1cf-4344-a198-7ed2624505a2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:43.246657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:43.326131 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10658e8e-f1cf-4344-a198-7ed2624505a2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:43.359101 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ceb22c-52e9-42d8-aa8c-543ecb847b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:43.359764 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ceb22c-52e9-42d8-aa8c-543ecb847b28 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:43.360052 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ceb22c-52e9-42d8-aa8c-543ecb847b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:43.360329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ceb22c-52e9-42d8-aa8c-543ecb847b28 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:43.361093 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ceb22c-52e9-42d8-aa8c-543ecb847b28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:43.361404 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ceb22c-52e9-42d8-aa8c-543ecb847b28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:43.362763 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ceb22c-52e9-42d8-aa8c-543ecb847b28 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:43.370704 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 631/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:23:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:43.377140 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38564461-8124-49a5-a8b3-1da3b82ed0db None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:43.444316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:43.477069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38564461-8124-49a5-a8b3-1da3b82ed0db None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:43.509442 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf33b0-ecfa-446a-b840-f75d3e275c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:43.510226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf33b0-ecfa-446a-b840-f75d3e275c9e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:43.510505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf33b0-ecfa-446a-b840-f75d3e275c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:43.510775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf33b0-ecfa-446a-b840-f75d3e275c9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:43.521541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf33b0-ecfa-446a-b840-f75d3e275c9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:43.522992 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf33b0-ecfa-446a-b840-f75d3e275c9e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:43.539480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-36bf33b0-ecfa-446a-b840-f75d3e275c9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:43.539480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:43.539480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:43.539480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:43.539480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:43.539480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:43.539480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:43.539480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:43.539480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:43.539480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:43.539480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:43.539480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:43.539480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:43.539480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:43.539480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:43.546509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:43.546509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:43.546509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:43.546509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:43.546509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:43.546509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:43.546509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:43.546509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:43.546509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:43.546509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:43.546509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:43.546509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:43.546509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:43.546509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:43.546509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:43.546509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:43.546509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:43.547568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:43.547568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:43.547568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:43.547568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:23:43.547568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 634/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:43.547910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ceb22c-52e9-42d8-aa8c-543ecb847b28 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:43.626642 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:43.645302 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ceb22c-52e9-42d8-aa8c-543ecb847b28 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:43.677828 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc11a8b-5906-4656-af89-b9f1ba45d91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:43.678476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc11a8b-5906-4656-af89-b9f1ba45d91a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:43.678758 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc11a8b-5906-4656-af89-b9f1ba45d91a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:43.679030 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc11a8b-5906-4656-af89-b9f1ba45d91a None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:43.679827 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc11a8b-5906-4656-af89-b9f1ba45d91a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:43.680134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc11a8b-5906-4656-af89-b9f1ba45d91a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:43.681523 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc11a8b-5906-4656-af89-b9f1ba45d91a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:43.693015 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 632/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:43.698846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf33b0-ecfa-446a-b840-f75d3e275c9e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:43.796958 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf33b0-ecfa-446a-b840-f75d3e275c9e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:43.816210 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:43.829301 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f386a4f-d9b7-4cd4-8d50-09ba689ffe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:43.830091 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f386a4f-d9b7-4cd4-8d50-09ba689ffe01 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:43.830371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f386a4f-d9b7-4cd4-8d50-09ba689ffe01 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:43.830646 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f386a4f-d9b7-4cd4-8d50-09ba689ffe01 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:43.837847 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f386a4f-d9b7-4cd4-8d50-09ba689ffe01 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:43.839278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f386a4f-d9b7-4cd4-8d50-09ba689ffe01 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:43.853101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-8f386a4f-d9b7-4cd4-8d50-09ba689ffe01 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:23:43.853101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:43.853101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:43.853101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:43.853101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:43.853101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:43.853101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:43.853101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:43.853101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:43.853101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:43.853101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:43.853101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:43.853101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:23:43.853101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:23:43.853101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:23:43.853101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:23:43.854072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:43.854072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:43.854072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:23:43.854072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:43.854072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:23:43.854072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:43.854072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:43.854072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:43.854072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:43.854072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:43.854072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:43.854072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:43.854072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:43.854072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:43.854072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:43.854072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:43.854072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:43.854072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:43.855101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:43.855101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:43.855101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:23:43.855101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:23:43.855101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:23:43.855101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:23:43.855101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:23:43.855101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:23:43.855101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:23:43.855101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:23:43.855101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 635/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:43.860603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc11a8b-5906-4656-af89-b9f1ba45d91a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:43.959984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc11a8b-5906-4656-af89-b9f1ba45d91a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:43.994200 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70041387-036f-4ecd-a0a4-3bdaa40259a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:43.994861 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70041387-036f-4ecd-a0a4-3bdaa40259a2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:43.995138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70041387-036f-4ecd-a0a4-3bdaa40259a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:43.995406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70041387-036f-4ecd-a0a4-3bdaa40259a2 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:43.996223 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70041387-036f-4ecd-a0a4-3bdaa40259a2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:43.996570 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70041387-036f-4ecd-a0a4-3bdaa40259a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:43.998013 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70041387-036f-4ecd-a0a4-3bdaa40259a2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:44.006920 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:44.014647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 633/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Jun 7 08:23:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:44.020311 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f386a4f-d9b7-4cd4-8d50-09ba689ffe01 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:44.118319 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f386a4f-d9b7-4cd4-8d50-09ba689ffe01 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:44.150034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec239bff-6e6c-437a-aeec-b1791564b70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:44.150833 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec239bff-6e6c-437a-aeec-b1791564b70d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:44.151104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec239bff-6e6c-437a-aeec-b1791564b70d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:44.151389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec239bff-6e6c-437a-aeec-b1791564b70d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:44.162165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec239bff-6e6c-437a-aeec-b1791564b70d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:44.163611 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec239bff-6e6c-437a-aeec-b1791564b70d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:44.181670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-ec239bff-6e6c-437a-aeec-b1791564b70d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:44.181670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:44.181670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:44.181670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:44.181670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:44.181670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:44.181670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:44.181670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:44.181670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:44.181670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:44.181670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:44.181670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:44.181670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:44.181670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:44.181670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:44.182664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:44.182664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:44.182664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:44.182664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:44.182664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:44.182664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:44.182664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:44.182664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:44.182664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:44.182664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:44.182664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:44.182664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:44.182664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:44.182664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:44.182664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:44.182664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:44.182664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:44.183555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:44.183555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:44.183555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:44.183555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:23:44.183555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 636/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:44.188768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70041387-036f-4ecd-a0a4-3bdaa40259a2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:44.287132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70041387-036f-4ecd-a0a4-3bdaa40259a2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:44.321695 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6acf24-e113-42ea-b369-144fd3f2ebb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:44.322352 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6acf24-e113-42ea-b369-144fd3f2ebb5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:44.322633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6acf24-e113-42ea-b369-144fd3f2ebb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:44.322907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6acf24-e113-42ea-b369-144fd3f2ebb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:44.323681 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6acf24-e113-42ea-b369-144fd3f2ebb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:44.324042 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6acf24-e113-42ea-b369-144fd3f2ebb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:44.325953 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6acf24-e113-42ea-b369-144fd3f2ebb5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:44.337572 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 634/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:44.343891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec239bff-6e6c-437a-aeec-b1791564b70d None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:44.375934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:44.443063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec239bff-6e6c-437a-aeec-b1791564b70d None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:44.476452 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae9d64-39fe-4abb-9efd-4406155bf82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:44.477299 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae9d64-39fe-4abb-9efd-4406155bf82b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:44.477575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae9d64-39fe-4abb-9efd-4406155bf82b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:44.477845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae9d64-39fe-4abb-9efd-4406155bf82b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:44.484253 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae9d64-39fe-4abb-9efd-4406155bf82b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:44.485754 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae9d64-39fe-4abb-9efd-4406155bf82b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:44.499492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-8cae9d64-39fe-4abb-9efd-4406155bf82b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:44.499492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:44.499492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:44.499492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:44.499492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:44.499492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:44.499492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:44.499492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:44.499492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:44.499492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:44.499492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:44.499492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:44.499492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:23:44.499492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:23:44.499492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:23:44.500682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:23:44.500682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:44.500682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:44.500682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:44.500682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:44.500682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:44.500682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:44.500682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:44.500682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:44.500682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:44.500682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:44.500682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:44.500682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:44.500682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:44.500682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:44.500682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:44.500682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:44.501980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:23:44.501980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:23:44.501980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:23:44.501980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:23:44.501980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:23:44.501980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:23:44.501980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:44.501980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:23:44.501980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 637/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:23:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:44.506480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6acf24-e113-42ea-b369-144fd3f2ebb5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:44.564699 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:44.605503 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6acf24-e113-42ea-b369-144fd3f2ebb5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:44.638614 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a21d93-415f-48b2-bba3-6f01ba834245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:44.639207 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a21d93-415f-48b2-bba3-6f01ba834245 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:44.639491 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a21d93-415f-48b2-bba3-6f01ba834245 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:44.639760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a21d93-415f-48b2-bba3-6f01ba834245 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:44.640538 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a21d93-415f-48b2-bba3-6f01ba834245 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:44.640841 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a21d93-415f-48b2-bba3-6f01ba834245 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:44.642300 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a21d93-415f-48b2-bba3-6f01ba834245 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:44.653812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 635/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:23:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:44.659749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae9d64-39fe-4abb-9efd-4406155bf82b None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:44.751698 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:44.760212 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae9d64-39fe-4abb-9efd-4406155bf82b None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:44.792891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf1d8d-20ee-44e6-b56d-8096eeb18616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:44.794342 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf1d8d-20ee-44e6-b56d-8096eeb18616 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:44.794630 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf1d8d-20ee-44e6-b56d-8096eeb18616 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:44.794906 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf1d8d-20ee-44e6-b56d-8096eeb18616 None admin] PATH_INFO: `/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/1fc0d7bd7e4547f4a8d90ffc2c2e1d03/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:44.797130 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf1d8d-20ee-44e6-b56d-8096eeb18616 None admin] RBAC: Authorizing `identity:create_grant(project_id=afc8781e5f864d4a9374300ee9b5193b, user_id=1fc0d7bd7e4547f4a8d90ffc2c2e1d03, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:44.798569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf1d8d-20ee-44e6-b56d-8096eeb18616 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:44.871549 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 638/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Jun 7 08:23:44 2022] PUT /identity/v3/projects/afc8781e5f864d4a9374300ee9b5193b/users/1fc0d7bd7e4547f4a8d90ffc2c2e1d03/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:23:47.067859 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf8dde-3384-4a13-81e9-0509e8ca12e1 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:47.068422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf8dde-3384-4a13-81e9-0509e8ca12e1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:47.068522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf8dde-3384-4a13-81e9-0509e8ca12e1 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:47.069866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 636/1274] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:47.075425 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093568a-ec5d-4a5a-a4bd-58aecea4e465 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:47.075872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093568a-ec5d-4a5a-a4bd-58aecea4e465 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:47.075987 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093568a-ec5d-4a5a-a4bd-58aecea4e465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:47.092549 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1093568a-ec5d-4a5a-a4bd-58aecea4e465 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:47.093766 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1093568a-ec5d-4a5a-a4bd-58aecea4e465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:47.160149 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 639/1275] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:47.184884 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf8dde-3384-4a13-81e9-0509e8ca12e1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:47.288870 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf8dde-3384-4a13-81e9-0509e8ca12e1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:47.308143 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:47.326960 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67e4e0-8324-4f6f-9502-9bb496446671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:47.327690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67e4e0-8324-4f6f-9502-9bb496446671 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:47.327991 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67e4e0-8324-4f6f-9502-9bb496446671 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:47.328452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67e4e0-8324-4f6f-9502-9bb496446671 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:47.340447 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67e4e0-8324-4f6f-9502-9bb496446671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 08:23:47.349072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-3f67e4e0-8324-4f6f-9502-9bb496446671 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:23:47.349072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:47.349072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:47.349072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:47.349072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:47.349072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:47.349072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:47.349072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:47.349072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:47.349072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:47.349072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:47.349072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:47.349072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 08:23:47.349072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 08:23:47.349072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 08:23:47.349072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 08:23:47.350316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 08:23:47.350316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 08:23:47.350316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 08:23:47.350316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 08:23:47.350316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:47.350316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:47.350316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:47.350316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:47.350316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:47.350316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:47.350316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:47.350316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:47.350316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:47.350316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:47.350316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:47.350316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:47.351495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:47.351495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:47.351495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:47.351495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:47.351495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 08:23:47.351495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 08:23:47.351495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 08:23:47.351495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 08:23:47.351495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 08:23:47.351495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 08:23:47.351495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 08:23:47.351495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:47.351495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 637/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Jun 7 08:23:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:47.356786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093568a-ec5d-4a5a-a4bd-58aecea4e465 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:47.451183 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:47.470422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093568a-ec5d-4a5a-a4bd-58aecea4e465 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:47.507593 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0cd5e-1f8f-4f4c-867f-573dddf53b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:47.508332 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0cd5e-1f8f-4f4c-867f-573dddf53b0c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:47.508622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0cd5e-1f8f-4f4c-867f-573dddf53b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:47.508899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0cd5e-1f8f-4f4c-867f-573dddf53b0c None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:47.509771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0cd5e-1f8f-4f4c-867f-573dddf53b0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:47.510025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0cd5e-1f8f-4f4c-867f-573dddf53b0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:47.511439 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0cd5e-1f8f-4f4c-867f-573dddf53b0c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:47.521510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 640/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Jun 7 08:23:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:23:47.527534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67e4e0-8324-4f6f-9502-9bb496446671 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:47.636795 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67e4e0-8324-4f6f-9502-9bb496446671 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:47.671296 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b1a3f-a153-4ea3-a33d-26f93530f6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:47.672101 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b1a3f-a153-4ea3-a33d-26f93530f6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:47.672387 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b1a3f-a153-4ea3-a33d-26f93530f6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:47.672664 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b1a3f-a153-4ea3-a33d-26f93530f6b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:47.684123 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b1a3f-a153-4ea3-a33d-26f93530f6b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:47.686146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b1a3f-a153-4ea3-a33d-26f93530f6b0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:47.703783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-728b1a3f-a153-4ea3-a33d-26f93530f6b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:47.703783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:47.703783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:47.703783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:47.703783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:47.703783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:47.703783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:47.703783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:47.703783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:47.703783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:47.703783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:47.703783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:47.703783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:47.703783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:47.703783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:47.708701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:47.708701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:47.708701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:47.708701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:47.708701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:47.708701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:47.708701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:47.708701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:47.708701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:47.708701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:47.708701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:47.708701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:47.708701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:47.708701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:47.708701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:47.708701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:47.708701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:47.709643 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:47.709643 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:47.709643 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:47.709643 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:47.709643 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 638/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 08:23:47.711579 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0cd5e-1f8f-4f4c-867f-573dddf53b0c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:47.815069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0cd5e-1f8f-4f4c-867f-573dddf53b0c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:47.847828 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0291ef-aa0e-4285-804c-84f0ed39f747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:47.850160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0291ef-aa0e-4285-804c-84f0ed39f747 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:47.850160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0291ef-aa0e-4285-804c-84f0ed39f747 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:47.850160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0291ef-aa0e-4285-804c-84f0ed39f747 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:47.850160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0291ef-aa0e-4285-804c-84f0ed39f747 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:47.850160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0291ef-aa0e-4285-804c-84f0ed39f747 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:47.851418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0291ef-aa0e-4285-804c-84f0ed39f747 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:47.862960 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 641/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:47.868784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b1a3f-a153-4ea3-a33d-26f93530f6b0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:47.899872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:47.966297 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b1a3f-a153-4ea3-a33d-26f93530f6b0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:47.998647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1ee1b-2734-4544-a778-d74cce75adf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:47.999430 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1ee1b-2734-4544-a778-d74cce75adf3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:47.999707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1ee1b-2734-4544-a778-d74cce75adf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:47.999977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1ee1b-2734-4544-a778-d74cce75adf3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:48.006804 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb1ee1b-2734-4544-a778-d74cce75adf3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:48.008228 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb1ee1b-2734-4544-a778-d74cce75adf3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:48.022803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-7eb1ee1b-2734-4544-a778-d74cce75adf3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:23:48.022803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:48.022803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:48.022803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:48.022803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:48.022803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:48.022803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:48.022803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:48.022803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:48.022803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:48.022803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:48.022803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:48.022803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 08:23:48.022803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 08:23:48.022803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 08:23:48.022803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 08:23:48.024299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:48.024299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:48.024299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 08:23:48.024299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:48.024299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 08:23:48.024299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 08:23:48.024299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:48.024299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:48.024299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:48.024299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:48.024299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:48.024299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:48.024299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:48.024299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:48.024299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:48.024299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:48.024299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:48.024299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:48.025522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:48.025522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:48.025522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 08:23:48.025522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 08:23:48.025522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 08:23:48.025522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 08:23:48.025522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 08:23:48.025522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 08:23:48.025522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 08:23:48.025522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:48.025522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 639/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:48.029858 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0291ef-aa0e-4285-804c-84f0ed39f747 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:48.033647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:48.126971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0291ef-aa0e-4285-804c-84f0ed39f747 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:48.159120 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c117c9e-c820-41dc-8428-3b1c4840f9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:48.159720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c117c9e-c820-41dc-8428-3b1c4840f9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:48.160004 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c117c9e-c820-41dc-8428-3b1c4840f9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:48.160277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c117c9e-c820-41dc-8428-3b1c4840f9f4 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:48.161133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c117c9e-c820-41dc-8428-3b1c4840f9f4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:48.161442 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c117c9e-c820-41dc-8428-3b1c4840f9f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:48.162912 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c117c9e-c820-41dc-8428-3b1c4840f9f4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:48.180485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 642/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Jun 7 08:23:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:48.186708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1ee1b-2734-4544-a778-d74cce75adf3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:48.272222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:48.281277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1ee1b-2734-4544-a778-d74cce75adf3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:48.315543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a4030-c0cd-4a62-8f4b-50e86cc489fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:48.316158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a4030-c0cd-4a62-8f4b-50e86cc489fd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:48.316437 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a4030-c0cd-4a62-8f4b-50e86cc489fd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:48.316708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a4030-c0cd-4a62-8f4b-50e86cc489fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:48.388196 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a4030-c0cd-4a62-8f4b-50e86cc489fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:48.389667 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a4030-c0cd-4a62-8f4b-50e86cc489fd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:48.480676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:48.491180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 640/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Jun 7 08:23:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 08:23:48.496871 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c117c9e-c820-41dc-8428-3b1c4840f9f4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:48.557516 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:48.594248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c117c9e-c820-41dc-8428-3b1c4840f9f4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:48.627173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6efdcb-f2de-412a-bd48-c3300827eff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:48.627951 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6efdcb-f2de-412a-bd48-c3300827eff4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:48.628226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6efdcb-f2de-412a-bd48-c3300827eff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:48.628495 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6efdcb-f2de-412a-bd48-c3300827eff4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:48.638415 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6efdcb-f2de-412a-bd48-c3300827eff4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:48.639842 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6efdcb-f2de-412a-bd48-c3300827eff4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:48.657136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-fc6efdcb-f2de-412a-bd48-c3300827eff4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:48.657136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:48.657136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:48.657136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:48.657136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:48.657136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:48.657136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:48.657136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:48.657136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:48.657136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:48.657136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:48.657136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:48.657136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 08:23:48.657136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 08:23:48.657136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 08:23:48.658205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 08:23:48.658205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:48.658205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:48.658205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:48.658205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:48.658205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:48.658205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:48.658205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:48.658205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:48.658205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:48.658205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:48.658205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:48.658205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:48.658205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:48.658205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:48.658205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:48.658205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:48.659464 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 08:23:48.659464 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 08:23:48.659464 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 08:23:48.659464 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:23:48.659464 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 643/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:23:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:48.663944 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a4030-c0cd-4a62-8f4b-50e86cc489fd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:48.759115 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a4030-c0cd-4a62-8f4b-50e86cc489fd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:48.793664 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bb33c-263f-448c-aa4f-8da628be4cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:48.794315 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bb33c-263f-448c-aa4f-8da628be4cbf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:48.794594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bb33c-263f-448c-aa4f-8da628be4cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:48.794871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bb33c-263f-448c-aa4f-8da628be4cbf None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:48.795572 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bb33c-263f-448c-aa4f-8da628be4cbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:48.795872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bb33c-263f-448c-aa4f-8da628be4cbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:48.797631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bb33c-263f-448c-aa4f-8da628be4cbf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:48.808690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 641/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Jun 7 08:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:48.814324 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6efdcb-f2de-412a-bd48-c3300827eff4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:48.909849 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6efdcb-f2de-412a-bd48-c3300827eff4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:48.923568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:48.943952 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d2ad5-f0ea-4fe4-81fa-3545a750c138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:48.944735 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d2ad5-f0ea-4fe4-81fa-3545a750c138 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:48.945038 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d2ad5-f0ea-4fe4-81fa-3545a750c138 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:48.945319 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d2ad5-f0ea-4fe4-81fa-3545a750c138 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:48.952401 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d2ad5-f0ea-4fe4-81fa-3545a750c138 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:48.953873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d2ad5-f0ea-4fe4-81fa-3545a750c138 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:48.968684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-499d2ad5-f0ea-4fe4-81fa-3545a750c138 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:48.968684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:48.968684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:48.968684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:48.968684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:48.968684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:48.968684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:48.968684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:48.968684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:48.968684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:48.968684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:48.968684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:48.968684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 08:23:48.968684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 08:23:48.968684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 08:23:48.969770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 08:23:48.969770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:48.969770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:48.969770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:48.969770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:48.969770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:48.969770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:48.969770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:48.969770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:48.969770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:48.969770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:48.969770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:48.969770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:48.969770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:48.969770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:48.969770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:48.969770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:48.970919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 08:23:48.970919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 08:23:48.970919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 08:23:48.970919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 08:23:48.970919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 08:23:48.970919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 08:23:48.970919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 08:23:48.970919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 07 08:23:48.970919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 644/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Jun 7 08:23:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:48.975909 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bb33c-263f-448c-aa4f-8da628be4cbf None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:49.025138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:49.074351 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bb33c-263f-448c-aa4f-8da628be4cbf None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:49.107274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4917e91-efc8-4c86-ba73-aa2f3a064c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:49.107926 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4917e91-efc8-4c86-ba73-aa2f3a064c87 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:49.108219 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4917e91-efc8-4c86-ba73-aa2f3a064c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:49.108498 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4917e91-efc8-4c86-ba73-aa2f3a064c87 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:49.109332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4917e91-efc8-4c86-ba73-aa2f3a064c87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:49.109646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4917e91-efc8-4c86-ba73-aa2f3a064c87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:49.111218 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4917e91-efc8-4c86-ba73-aa2f3a064c87 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:49.123598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 642/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Jun 7 08:23:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:49.144431 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d2ad5-f0ea-4fe4-81fa-3545a750c138 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:49.227007 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d2ad5-f0ea-4fe4-81fa-3545a750c138 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:49.259837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90304774-b0d1-4862-a506-4c97987bc962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:49.260478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90304774-b0d1-4862-a506-4c97987bc962 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:49.260760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90304774-b0d1-4862-a506-4c97987bc962 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:49.261059 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90304774-b0d1-4862-a506-4c97987bc962 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:49.261880 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90304774-b0d1-4862-a506-4c97987bc962 None admin] RBAC: Adding query filter params (role.id=8c63c00418a346b29f7db3f87a404e70, scope.project.id=afc8781e5f864d4a9374300ee9b5193b, user.id=1fc0d7bd7e4547f4a8d90ffc2c2e1d03) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:49.262191 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90304774-b0d1-4862-a506-4c97987bc962 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:49.264180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90304774-b0d1-4862-a506-4c97987bc962 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:49.269937 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 645/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Jun 7 08:23:49 2022] GET /identity/v3/role_assignments?user.id=1fc0d7bd7e4547f4a8d90ffc2c2e1d03&scope.project.id=afc8781e5f864d4a9374300ee9b5193b&role.id=8c63c00418a346b29f7db3f87a404e70 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:51.478960 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb00681-745e-4eb0-a717-a259423d23e0 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:51.479475 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb00681-745e-4eb0-a717-a259423d23e0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:51.479475 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb00681-745e-4eb0-a717-a259423d23e0 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:51.480763 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 643/1288] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:51.485959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f158bc-d7b9-44af-b71e-cb92423661b8 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:51.486396 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f158bc-d7b9-44af-b71e-cb92423661b8 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:51.486606 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f158bc-d7b9-44af-b71e-cb92423661b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:51.503514 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e6f158bc-d7b9-44af-b71e-cb92423661b8 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:51.504677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e6f158bc-d7b9-44af-b71e-cb92423661b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:51.533700 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:51.569671 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 646/1289] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:51.592351 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb00681-745e-4eb0-a717-a259423d23e0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:51.695656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb00681-745e-4eb0-a717-a259423d23e0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:51.706631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:51.728505 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d0a72-ea40-4109-86e0-96295cd3dc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:51.731933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d0a72-ea40-4109-86e0-96295cd3dc42 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:51.731933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d0a72-ea40-4109-86e0-96295cd3dc42 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:51.731933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d0a72-ea40-4109-86e0-96295cd3dc42 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:51.731933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d0a72-ea40-4109-86e0-96295cd3dc42 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:51.731933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d0a72-ea40-4109-86e0-96295cd3dc42 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:51.743332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a49d0a72-ea40-4109-86e0-96295cd3dc42 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 08:23:51.743332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:51.743332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:23:51.743332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:23:51.743332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:23:51.743332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:23:51.743332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:23:51.743332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:51.743332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 08:23:51.743332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:51.743332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:23:51.743332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:51.743332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:51.743332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:51.743332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:51.743332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:51.743332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:51.744569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:51.744569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:51.744569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:51.744569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:51.744569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:51.744569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:51.744569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:23:51.744569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:23:51.744569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:23:51.744569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:23:51.744569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:51.744569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:51.744569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:51.744569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:51.744569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:51.744569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:51.745707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:51.745707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:51.745707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:51.745707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:51.745707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:51.745707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:51.745707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:51.745707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:51.745707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:51.745707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:51.745707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:23:51.745707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:51.745707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 08:23:51.745707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:51.745707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 644/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jun 7 08:23:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:51.751195 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f158bc-d7b9-44af-b71e-cb92423661b8 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:51.847694 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f158bc-d7b9-44af-b71e-cb92423661b8 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:51.883352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bb20a-c178-4097-95b9-e734c2b36515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:51.884008 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bb20a-c178-4097-95b9-e734c2b36515 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:51.884285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bb20a-c178-4097-95b9-e734c2b36515 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:51.884556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bb20a-c178-4097-95b9-e734c2b36515 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:51.885285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bb20a-c178-4097-95b9-e734c2b36515 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:51.885591 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bb20a-c178-4097-95b9-e734c2b36515 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:51.886956 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bb20a-c178-4097-95b9-e734c2b36515 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:51.891756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 647/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Jun 7 08:23:51 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:51.897578 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d0a72-ea40-4109-86e0-96295cd3dc42 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:51.993866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d0a72-ea40-4109-86e0-96295cd3dc42 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:52.026758 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd04467-67ff-40d5-a83f-eb003811c14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:52.027413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd04467-67ff-40d5-a83f-eb003811c14d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:52.027696 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd04467-67ff-40d5-a83f-eb003811c14d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:52.027969 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd04467-67ff-40d5-a83f-eb003811c14d None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:52.028670 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd04467-67ff-40d5-a83f-eb003811c14d None admin] RBAC: Adding query filter params (type=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:52.028974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd04467-67ff-40d5-a83f-eb003811c14d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:52.030363 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd04467-67ff-40d5-a83f-eb003811c14d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:52.034937 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 645/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Jun 7 08:23:51 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:54.260510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6b932-f319-4521-a963-4936c6a553ee None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:54.261076 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6b932-f319-4521-a963-4936c6a553ee None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:54.261076 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6b932-f319-4521-a963-4936c6a553ee None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:54.262972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 648/1293] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:23:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:54.268344 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f188825-d945-48ad-a84b-9a38072eb353 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:54.268759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f188825-d945-48ad-a84b-9a38072eb353 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:54.268897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f188825-d945-48ad-a84b-9a38072eb353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:54.285865 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8f188825-d945-48ad-a84b-9a38072eb353 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:54.287046 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8f188825-d945-48ad-a84b-9a38072eb353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:54.315574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:54.351689 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 646/1294] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:54.374930 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6b932-f319-4521-a963-4936c6a553ee None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:54.406063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:54.475895 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6b932-f319-4521-a963-4936c6a553ee None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:54.508536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607ed46-677c-4dd8-94f7-83122d406fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:54.509222 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607ed46-677c-4dd8-94f7-83122d406fb5 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:54.509504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607ed46-677c-4dd8-94f7-83122d406fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:54.509778 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607ed46-677c-4dd8-94f7-83122d406fb5 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:54.510545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607ed46-677c-4dd8-94f7-83122d406fb5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:54.511910 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607ed46-677c-4dd8-94f7-83122d406fb5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:54.580868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 649/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Jun 7 08:23:54 2022] POST /identity/v3/services => generated 246 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:23:56.822786 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da1c52-0099-4bc1-83d3-0f50c6803ccf None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:56.823350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da1c52-0099-4bc1-83d3-0f50c6803ccf None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:56.823350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da1c52-0099-4bc1-83d3-0f50c6803ccf None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:56.824721 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 647/1296] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:56.830510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c49581-f25d-475d-b092-f97ee0d4ddb0 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:56.832873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c49581-f25d-475d-b092-f97ee0d4ddb0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:56.832873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c49581-f25d-475d-b092-f97ee0d4ddb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:56.847324 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-42c49581-f25d-475d-b092-f97ee0d4ddb0 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:56.848487 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-42c49581-f25d-475d-b092-f97ee0d4ddb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:56.918972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 650/1297] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:56 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:56.942164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da1c52-0099-4bc1-83d3-0f50c6803ccf None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:57.042240 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da1c52-0099-4bc1-83d3-0f50c6803ccf None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:57.075754 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4ac8b-b1f3-4e73-b8ea-da2c40425400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:57.076471 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4ac8b-b1f3-4e73-b8ea-da2c40425400 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:57.076751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4ac8b-b1f3-4e73-b8ea-da2c40425400 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:57.077044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4ac8b-b1f3-4e73-b8ea-da2c40425400 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:57.077686 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4ac8b-b1f3-4e73-b8ea-da2c40425400 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:57.079040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4ac8b-b1f3-4e73-b8ea-da2c40425400 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:57.090395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-92c4ac8b-b1f3-4e73-b8ea-da2c40425400 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 08:23:57.090395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:57.090395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:23:57.090395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:23:57.090395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:23:57.090395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:23:57.090395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:23:57.090395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:57.090395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 08:23:57.090395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:57.090395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:23:57.090395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:57.090395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:23:57.090395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:23:57.090395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:23:57.090395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:23:57.090395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:23:57.091560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:23:57.091560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:23:57.091560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:23:57.091560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:23:57.091560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:23:57.091560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:23:57.091560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:23:57.091560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:23:57.091560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:23:57.091560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:23:57.091560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:23:57.091560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:23:57.091560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:23:57.091560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:23:57.091560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:23:57.091560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:23:57.092416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:23:57.092416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:23:57.092416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:23:57.092416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:23:57.092416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:23:57.092416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:23:57.092416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:23:57.092416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:23:57.092416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:23:57.092416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:23:57.092416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:23:57.092416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:23:57.092416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 08:23:57.092416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:23:57.092416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 648/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jun 7 08:23:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:23:57.097717 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c49581-f25d-475d-b092-f97ee0d4ddb0 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:57.116231 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:57.195458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c49581-f25d-475d-b092-f97ee0d4ddb0 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:57.264973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca825e-0b5c-4764-a467-ce2a841d71e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:57.264973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca825e-0b5c-4764-a467-ce2a841d71e7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:57.264973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca825e-0b5c-4764-a467-ce2a841d71e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:57.264973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca825e-0b5c-4764-a467-ce2a841d71e7 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:57.265417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca825e-0b5c-4764-a467-ce2a841d71e7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:57.265417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca825e-0b5c-4764-a467-ce2a841d71e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:57.265417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca825e-0b5c-4764-a467-ce2a841d71e7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:57.265417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 651/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Jun 7 08:23:57 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:57.265751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4ac8b-b1f3-4e73-b8ea-da2c40425400 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:57.275087 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:57.340834 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4ac8b-b1f3-4e73-b8ea-da2c40425400 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:57.374428 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a427b-f052-4de5-bcb1-2b73db04187e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:57.375092 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a427b-f052-4de5-bcb1-2b73db04187e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:57.375368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a427b-f052-4de5-bcb1-2b73db04187e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:57.375638 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a427b-f052-4de5-bcb1-2b73db04187e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:57.376403 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a427b-f052-4de5-bcb1-2b73db04187e None admin] RBAC: Adding query filter params (interface=public, service_id=d7f3ca4588f048238f5431c85e3b270b, region_id=RegionOne) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:23:57.376710 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a427b-f052-4de5-bcb1-2b73db04187e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:57.378089 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a427b-f052-4de5-bcb1-2b73db04187e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:57.382963 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 649/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Jun 7 08:23:57 2022] GET /identity/v3/endpoints?service_id=d7f3ca4588f048238f5431c85e3b270b&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:23:57.389165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca825e-0b5c-4764-a467-ce2a841d71e7 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:57.468071 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:57.484988 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca825e-0b5c-4764-a467-ce2a841d71e7 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:23:57.518178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04636ad5-e11c-4a78-b704-441aa94826a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:23:57.518835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04636ad5-e11c-4a78-b704-441aa94826a7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:57.519118 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04636ad5-e11c-4a78-b704-441aa94826a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:57.519393 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04636ad5-e11c-4a78-b704-441aa94826a7 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:57.520071 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04636ad5-e11c-4a78-b704-441aa94826a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:23:57.521587 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04636ad5-e11c-4a78-b704-441aa94826a7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:23:57.528145 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 652/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Jun 7 08:23:57 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:23:59.768650 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bf346-73f1-4eab-ac77-831ea2c9cf6c None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:59.769239 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bf346-73f1-4eab-ac77-831ea2c9cf6c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:59.769239 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bf346-73f1-4eab-ac77-831ea2c9cf6c None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:59.770586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 650/1302] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:23:59.775736 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e4bb6-6a03-464e-848c-eac77ef5eb29 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:23:59.776175 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e4bb6-6a03-464e-848c-eac77ef5eb29 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:23:59.776291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e4bb6-6a03-464e-848c-eac77ef5eb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:23:59.792873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-8d0e4bb6-6a03-464e-848c-eac77ef5eb29 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:23:59.794716 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8d0e4bb6-6a03-464e-848c-eac77ef5eb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:23:59.858722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 653/1303] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:23:59 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 08:23:59.880956 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bf346-73f1-4eab-ac77-831ea2c9cf6c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:23:59.972081 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:23:59.978312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bf346-73f1-4eab-ac77-831ea2c9cf6c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:00.010737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565f00c-f7a8-4417-a821-1c7913b408c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:00.011524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565f00c-f7a8-4417-a821-1c7913b408c7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:00.011808 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565f00c-f7a8-4417-a821-1c7913b408c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:00.012082 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565f00c-f7a8-4417-a821-1c7913b408c7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:00.013314 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565f00c-f7a8-4417-a821-1c7913b408c7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:00.014663 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565f00c-f7a8-4417-a821-1c7913b408c7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:00.027012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1565f00c-f7a8-4417-a821-1c7913b408c7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 08:24:00.027012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:24:00.027012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 08:24:00.027012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 08:24:00.027012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 08:24:00.027012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 08:24:00.027012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 08:24:00.027012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:24:00.027012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 08:24:00.027012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:24:00.027012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 08:24:00.027012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:24:00.027012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 08:24:00.027012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 08:24:00.027012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 08:24:00.027012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 08:24:00.027012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 08:24:00.028709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 08:24:00.028709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 08:24:00.028709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 08:24:00.028709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 08:24:00.028709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 08:24:00.028709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 08:24:00.028709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 08:24:00.028709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 08:24:00.028709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 08:24:00.028709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 08:24:00.028709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 08:24:00.028709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 08:24:00.028709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 08:24:00.028709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 08:24:00.028709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 08:24:00.028709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 08:24:00.029658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 08:24:00.029658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 07 08:24:00.029658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 08:24:00.029658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 07 08:24:00.029658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 08:24:00.029658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 08:24:00.029658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 08:24:00.029658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 07 08:24:00.029658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 08:24:00.029658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 07 08:24:00.029658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 08:24:00.029658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 08:24:00.029658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 08:24:00.029658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 07 08:24:00.029658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 651/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Jun 7 08:23:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 08:24:00.034593 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e4bb6-6a03-464e-848c-eac77ef5eb29 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:00.108490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:00.130132 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e4bb6-6a03-464e-848c-eac77ef5eb29 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:00.163726 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebcb1e-3c86-4c2e-9a74-131060e5badc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:00.164369 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebcb1e-3c86-4c2e-9a74-131060e5badc None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:00.164647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebcb1e-3c86-4c2e-9a74-131060e5badc None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:00.164950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebcb1e-3c86-4c2e-9a74-131060e5badc None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:00.165745 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ebcb1e-3c86-4c2e-9a74-131060e5badc None admin] RBAC: Adding query filter params (name=placement) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:24:00.166045 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ebcb1e-3c86-4c2e-9a74-131060e5badc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:00.167440 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ebcb1e-3c86-4c2e-9a74-131060e5badc None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:00.172802 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 654/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Jun 7 08:24:00 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:24:00.178550 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565f00c-f7a8-4417-a821-1c7913b408c7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:24:00.275569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565f00c-f7a8-4417-a821-1c7913b408c7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:24:00.308049 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80185099-84dd-4489-a1aa-52ad313fe5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:24:00.308639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80185099-84dd-4489-a1aa-52ad313fe5c0 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:24:00.308922 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80185099-84dd-4489-a1aa-52ad313fe5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:24:00.309248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80185099-84dd-4489-a1aa-52ad313fe5c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:24:00.309972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80185099-84dd-4489-a1aa-52ad313fe5c0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:24:00.311376 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80185099-84dd-4489-a1aa-52ad313fe5c0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:24:00.391993 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 652/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Jun 7 08:24:00 2022] POST /identity/v3/endpoints => generated 331 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:24:10.169434 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=30 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:24:10.386944 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:24.682787 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2314add-b250-4b4b-b3f7-89024eadb365 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:24.683696 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2314add-b250-4b4b-b3f7-89024eadb365 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:24.684121 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2314add-b250-4b4b-b3f7-89024eadb365 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:24.691072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7622f21-5bcd-47db-8645-75c036920db1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:24.691872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7622f21-5bcd-47db-8645-75c036920db1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:24.692299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7622f21-5bcd-47db-8645-75c036920db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:24.695758 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 655/1307] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:29:24 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:24.799955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a7622f21-5bcd-47db-8645-75c036920db1 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:24.801810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a7622f21-5bcd-47db-8645-75c036920db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:24.965590 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 653/1308] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:29:25.007022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf4783-bb76-4738-b3b4-e1eb4fc3ff5c None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:25.007544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf4783-bb76-4738-b3b4-e1eb4fc3ff5c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:25.014343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf4783-bb76-4738-b3b4-e1eb4fc3ff5c None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:25.015835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 656/1309] 10.222.0.17 () {54 vars in 964 bytes} [Tue Jun 7 08:29:25 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:25.025278 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c703c12-d2f4-4216-b26f-8000d1e4f3a0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:25.025722 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c703c12-d2f4-4216-b26f-8000d1e4f3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:25.026106 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c703c12-d2f4-4216-b26f-8000d1e4f3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:25.119819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6c703c12-d2f4-4216-b26f-8000d1e4f3a0 None None] MFA Rules not processed for user `4a87f8bb06ee4a3ba7017422f4f1943e`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:25.121272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6c703c12-d2f4-4216-b26f-8000d1e4f3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:25.144255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:25.220847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 654/1310] 10.222.0.17 () {58 vars in 1061 bytes} [Tue Jun 7 08:29:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:25.230872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf4783-bb76-4738-b3b4-e1eb4fc3ff5c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:25.232885 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-41bf4783-bb76-4738-b3b4-e1eb4fc3ff5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:25.346556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf4783-bb76-4738-b3b4-e1eb4fc3ff5c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:25.429951 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa30b4-0b56-4500-acfb-be2c048404c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:25.430621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa30b4-0b56-4500-acfb-be2c048404c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:25.430904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa30b4-0b56-4500-acfb-be2c048404c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:25.431181 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa30b4-0b56-4500-acfb-be2c048404c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:25.432839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1efa30b4-0b56-4500-acfb-be2c048404c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:25.490523 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:25.551072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa30b4-0b56-4500-acfb-be2c048404c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:25.552670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa30b4-0b56-4500-acfb-be2c048404c5 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:25.573207 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 657/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:28.158254 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe5713-aead-45ae-9822-85306ae3cbb9 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:28.158756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe5713-aead-45ae-9822-85306ae3cbb9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:28.158852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe5713-aead-45ae-9822-85306ae3cbb9 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:28.164357 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 655/1312] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:29:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:28.165859 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3a58a-9d6a-4cba-8899-ad3154611e77 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:28.166296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3a58a-9d6a-4cba-8899-ad3154611e77 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:28.166296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3a58a-9d6a-4cba-8899-ad3154611e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:28.262611 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a8d3a58a-9d6a-4cba-8899-ad3154611e77 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:28.264656 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a8d3a58a-9d6a-4cba-8899-ad3154611e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:28.377036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 658/1313] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:28.414940 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe5713-aead-45ae-9822-85306ae3cbb9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:28.416488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d5fe5713-aead-45ae-9822-85306ae3cbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:28.537247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe5713-aead-45ae-9822-85306ae3cbb9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:28.611022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:28.613546 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbcdf37-d791-4d48-a74e-bc4f88fbbe1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:28.614189 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbcdf37-d791-4d48-a74e-bc4f88fbbe1f service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:28.614468 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbcdf37-d791-4d48-a74e-bc4f88fbbe1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:28.614738 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbcdf37-d791-4d48-a74e-bc4f88fbbe1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:28.616354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0cbcdf37-d791-4d48-a74e-bc4f88fbbe1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:28.721462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbcdf37-d791-4d48-a74e-bc4f88fbbe1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:28.722886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbcdf37-d791-4d48-a74e-bc4f88fbbe1f service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:28.741389 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 656/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:31.458610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cb8a6-7f67-422b-b9d7-eaf3526a245e None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:31.459156 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cb8a6-7f67-422b-b9d7-eaf3526a245e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:31.459156 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cb8a6-7f67-422b-b9d7-eaf3526a245e None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:31.460450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 659/1315] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:31.465519 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a2612-8cf8-45e9-a999-b96d1edcd640 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:31.465914 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a2612-8cf8-45e9-a999-b96d1edcd640 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:31.466069 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a2612-8cf8-45e9-a999-b96d1edcd640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:31.530283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3d1a2612-8cf8-45e9-a999-b96d1edcd640 None None] MFA Rules not processed for user `73798cb88b144a3db46809da81f534cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:31.531468 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3d1a2612-8cf8-45e9-a999-b96d1edcd640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:31.581014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:31.624180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 657/1316] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:29:31 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:31.663472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9cb8a6-7f67-422b-b9d7-eaf3526a245e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:31.665021 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ef9cb8a6-7f67-422b-b9d7-eaf3526a245e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:31.675561 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:31.766806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9cb8a6-7f67-422b-b9d7-eaf3526a245e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:31.963934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c31183-2a5e-4ce2-a2ac-56ad078ab42f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:31.964598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c31183-2a5e-4ce2-a2ac-56ad078ab42f service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:31.964876 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c31183-2a5e-4ce2-a2ac-56ad078ab42f service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:31.965182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c31183-2a5e-4ce2-a2ac-56ad078ab42f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:31.967528 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-98c31183-2a5e-4ce2-a2ac-56ad078ab42f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:32.026988 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:32.072493 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c31183-2a5e-4ce2-a2ac-56ad078ab42f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:32.073946 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c31183-2a5e-4ce2-a2ac-56ad078ab42f service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:32.091465 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 660/1317] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:31 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:35.074578 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7eb8c-816f-4ddf-ac56-f38a9439b24b None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:35.075125 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7eb8c-816f-4ddf-ac56-f38a9439b24b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:35.075125 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7eb8c-816f-4ddf-ac56-f38a9439b24b None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:35.076475 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 658/1318] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:35.082194 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afefa4d-f98b-45a9-b237-72b8c8b03d5b None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:35.082640 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afefa4d-f98b-45a9-b237-72b8c8b03d5b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:35.082734 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afefa4d-f98b-45a9-b237-72b8c8b03d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:35.145039 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7afefa4d-f98b-45a9-b237-72b8c8b03d5b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:35.146150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7afefa4d-f98b-45a9-b237-72b8c8b03d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:35.238392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 661/1319] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:29:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:35.292415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a52e0-0fda-49e8-91b2-b38de41dbb64 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:35.292702 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a52e0-0fda-49e8-91b2-b38de41dbb64 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:35.292985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a52e0-0fda-49e8-91b2-b38de41dbb64 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:35.294391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 659/1320] 10.222.0.17 () {54 vars in 964 bytes} [Tue Jun 7 08:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:35.300444 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f1b44-1b97-4e7b-a618-60725e56ebbe None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:35.300736 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f1b44-1b97-4e7b-a618-60725e56ebbe None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:35.301063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f1b44-1b97-4e7b-a618-60725e56ebbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:35.321919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:35.373770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-407f1b44-1b97-4e7b-a618-60725e56ebbe None None] MFA Rules not processed for user `4a87f8bb06ee4a3ba7017422f4f1943e`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:35.375485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-407f1b44-1b97-4e7b-a618-60725e56ebbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:35.449632 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 662/1321] 10.222.0.17 () {58 vars in 1061 bytes} [Tue Jun 7 08:29:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:35.459451 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a52e0-0fda-49e8-91b2-b38de41dbb64 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:35.460952 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4c0a52e0-0fda-49e8-91b2-b38de41dbb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:35.559727 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a52e0-0fda-49e8-91b2-b38de41dbb64 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:35.631849 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c453b-03ff-45ed-8fe6-b8fda71103ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:35.632484 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c453b-03ff-45ed-8fe6-b8fda71103ed service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:35.632763 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c453b-03ff-45ed-8fe6-b8fda71103ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:35.633057 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c453b-03ff-45ed-8fe6-b8fda71103ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:35.634700 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f04c453b-03ff-45ed-8fe6-b8fda71103ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:35.641905 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:35.739216 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c453b-03ff-45ed-8fe6-b8fda71103ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:35.740565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c453b-03ff-45ed-8fe6-b8fda71103ed service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:35.758210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 660/1322] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:39.404782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4de15-a1f2-4310-8b11-59e7df087280 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:39.405989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4de15-a1f2-4310-8b11-59e7df087280 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:39.405989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4de15-a1f2-4310-8b11-59e7df087280 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:39.407263 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 663/1323] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:29:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:39.412584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c32c6-f78f-4795-8e76-4a04e136121a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:39.412984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c32c6-f78f-4795-8e76-4a04e136121a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:39.413293 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c32c6-f78f-4795-8e76-4a04e136121a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:39.477447 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-150c32c6-f78f-4795-8e76-4a04e136121a None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:39.478542 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-150c32c6-f78f-4795-8e76-4a04e136121a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:39.538099 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:39.575530 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 661/1324] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:39.611464 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4de15-a1f2-4310-8b11-59e7df087280 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:39.613080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4cb4de15-a1f2-4310-8b11-59e7df087280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:39.715891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4de15-a1f2-4310-8b11-59e7df087280 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:39.729040 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:39.789435 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af6f69-13eb-4df1-b875-91c1805b6d54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:39.790079 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af6f69-13eb-4df1-b875-91c1805b6d54 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:39.790355 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af6f69-13eb-4df1-b875-91c1805b6d54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:39.790627 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af6f69-13eb-4df1-b875-91c1805b6d54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:39.792528 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e3af6f69-13eb-4df1-b875-91c1805b6d54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:39.900000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af6f69-13eb-4df1-b875-91c1805b6d54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:39.901493 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af6f69-13eb-4df1-b875-91c1805b6d54 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:39.919381 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 664/1325] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:42.214793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fff74b-082f-4082-975f-1caa27fbb40f None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:42.215333 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fff74b-082f-4082-975f-1caa27fbb40f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:42.215460 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fff74b-082f-4082-975f-1caa27fbb40f None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:42.216725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 662/1326] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:29:42.222566 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907179a-3739-45cb-bdbe-b0ac89bbc61c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:42.222971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907179a-3739-45cb-bdbe-b0ac89bbc61c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:42.223119 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907179a-3739-45cb-bdbe-b0ac89bbc61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:42.269495 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:42.287447 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7907179a-3739-45cb-bdbe-b0ac89bbc61c None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:42.288642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7907179a-3739-45cb-bdbe-b0ac89bbc61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:42.381768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 665/1327] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:29:42.418667 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fff74b-082f-4082-975f-1caa27fbb40f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:42.420210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e0fff74b-082f-4082-975f-1caa27fbb40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:42.521818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fff74b-082f-4082-975f-1caa27fbb40f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:42.594625 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583dd6a-e928-4449-8d96-2a0523dd9886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:42.595273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583dd6a-e928-4449-8d96-2a0523dd9886 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:42.595554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583dd6a-e928-4449-8d96-2a0523dd9886 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:42.595828 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583dd6a-e928-4449-8d96-2a0523dd9886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:42.597489 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6583dd6a-e928-4449-8d96-2a0523dd9886 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:42.613891 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:42.706136 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583dd6a-e928-4449-8d96-2a0523dd9886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:42.707573 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583dd6a-e928-4449-8d96-2a0523dd9886 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:42.725634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 663/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:45.002765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc92ed-2473-43a8-b567-f30bced9a96d None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:45.003281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc92ed-2473-43a8-b567-f30bced9a96d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:45.003392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc92ed-2473-43a8-b567-f30bced9a96d None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:45.004665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 666/1329] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:45.010061 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bcb2f-edd1-436c-81de-e7676b1d7baf None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:45.010489 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bcb2f-edd1-436c-81de-e7676b1d7baf None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:45.010613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bcb2f-edd1-436c-81de-e7676b1d7baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:45.073466 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d41bcb2f-edd1-436c-81de-e7676b1d7baf None None] MFA Rules not processed for user `73798cb88b144a3db46809da81f534cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:45.074653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d41bcb2f-edd1-436c-81de-e7676b1d7baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:45.137697 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:45.169896 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 664/1330] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:29:45 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:45.207305 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc92ed-2473-43a8-b567-f30bced9a96d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:45.208840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f2cc92ed-2473-43a8-b567-f30bced9a96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:45.314276 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc92ed-2473-43a8-b567-f30bced9a96d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:45.337490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:45.390661 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5296e-6588-4015-9579-2c54f429d834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:45.391301 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5296e-6588-4015-9579-2c54f429d834 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:45.391575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5296e-6588-4015-9579-2c54f429d834 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:45.391867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5296e-6588-4015-9579-2c54f429d834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:45.393530 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-beb5296e-6588-4015-9579-2c54f429d834 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:45.500010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5296e-6588-4015-9579-2c54f429d834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:45.501425 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5296e-6588-4015-9579-2c54f429d834 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:45.519403 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 667/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:45 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:48.221592 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd52432-9915-427a-beb6-a0f2e2427c28 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:48.222093 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd52432-9915-427a-beb6-a0f2e2427c28 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:48.222200 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd52432-9915-427a-beb6-a0f2e2427c28 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:48.223500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 665/1332] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:48.228761 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35627a69-7dab-44da-aa85-8a0f47827ba9 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:48.229331 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35627a69-7dab-44da-aa85-8a0f47827ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:48.229496 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35627a69-7dab-44da-aa85-8a0f47827ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:48.291250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:48.294283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-35627a69-7dab-44da-aa85-8a0f47827ba9 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:48.295516 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-35627a69-7dab-44da-aa85-8a0f47827ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:48.393367 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 668/1333] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:48.430362 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd52432-9915-427a-beb6-a0f2e2427c28 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:48.431886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1fd52432-9915-427a-beb6-a0f2e2427c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:48.536514 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd52432-9915-427a-beb6-a0f2e2427c28 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:48.608406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65efea41-f229-4f2b-99d4-e6d688df96a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:48.609089 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65efea41-f229-4f2b-99d4-e6d688df96a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:48.609375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65efea41-f229-4f2b-99d4-e6d688df96a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:48.609657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65efea41-f229-4f2b-99d4-e6d688df96a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:48.611310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-65efea41-f229-4f2b-99d4-e6d688df96a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:48.638231 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:48.721818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65efea41-f229-4f2b-99d4-e6d688df96a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:48.723301 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65efea41-f229-4f2b-99d4-e6d688df96a2 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:48.741702 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 666/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:51.032101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd1a42-f3c4-4392-8f7d-809f6f4aa964 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:51.032609 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd1a42-f3c4-4392-8f7d-809f6f4aa964 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:51.032718 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd1a42-f3c4-4392-8f7d-809f6f4aa964 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:51.034043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 669/1335] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:51.039287 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b371a-a9e9-433b-a24a-1295c33f8aa0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:51.039723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b371a-a9e9-433b-a24a-1295c33f8aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:51.039836 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b371a-a9e9-433b-a24a-1295c33f8aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:51.102988 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fc6b371a-a9e9-433b-a24a-1295c33f8aa0 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:51.104178 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fc6b371a-a9e9-433b-a24a-1295c33f8aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:51.175526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:51.197518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 667/1336] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:51.237856 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd1a42-f3c4-4392-8f7d-809f6f4aa964 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:51.239386 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4dfd1a42-f3c4-4392-8f7d-809f6f4aa964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:51.345718 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd1a42-f3c4-4392-8f7d-809f6f4aa964 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:51.372811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:51.418264 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680e2a3-266f-4b06-b1f6-80c6794d8aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:51.418896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680e2a3-266f-4b06-b1f6-80c6794d8aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:51.419172 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680e2a3-266f-4b06-b1f6-80c6794d8aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:51.419440 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680e2a3-266f-4b06-b1f6-80c6794d8aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:51.421109 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0680e2a3-266f-4b06-b1f6-80c6794d8aa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:51.524054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680e2a3-266f-4b06-b1f6-80c6794d8aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:51.525844 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680e2a3-266f-4b06-b1f6-80c6794d8aa0 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:51.544462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 670/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:54.314404 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1445a5b-3137-4631-9ab0-f9603535f7af None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:54.314932 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1445a5b-3137-4631-9ab0-f9603535f7af None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:54.314932 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1445a5b-3137-4631-9ab0-f9603535f7af None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:54.316219 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 668/1338] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:54.321394 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aecbc6-5d71-4756-8a02-81d0181f74a5 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:54.321806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aecbc6-5d71-4756-8a02-81d0181f74a5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:54.321914 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aecbc6-5d71-4756-8a02-81d0181f74a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:54.384685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a5aecbc6-5d71-4756-8a02-81d0181f74a5 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:54.385912 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a5aecbc6-5d71-4756-8a02-81d0181f74a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:54.394398 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:54.479671 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 671/1339] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:29:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:29:54.515259 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1445a5b-3137-4631-9ab0-f9603535f7af None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:54.516771 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c1445a5b-3137-4631-9ab0-f9603535f7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:54.615992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1445a5b-3137-4631-9ab0-f9603535f7af None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:54.687377 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e1a09-a2cf-4c7c-94a9-eb370236bca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:54.688017 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e1a09-a2cf-4c7c-94a9-eb370236bca8 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:54.688296 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e1a09-a2cf-4c7c-94a9-eb370236bca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:54.688595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e1a09-a2cf-4c7c-94a9-eb370236bca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:54.690279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7b4e1a09-a2cf-4c7c-94a9-eb370236bca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:54.727451 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:54.799630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e1a09-a2cf-4c7c-94a9-eb370236bca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:54.801095 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e1a09-a2cf-4c7c-94a9-eb370236bca8 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:54.819058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 669/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:29:59.291727 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ed7cc-43c9-4584-96c9-76fdfebfae0f None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:59.292208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ed7cc-43c9-4584-96c9-76fdfebfae0f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:59.292290 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ed7cc-43c9-4584-96c9-76fdfebfae0f None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:59.297061 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 672/1341] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:29:59.299299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4257477-f858-49c9-84d1-7d1f50a6f1c4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:59.299761 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4257477-f858-49c9-84d1-7d1f50a6f1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:59.299847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4257477-f858-49c9-84d1-7d1f50a6f1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:59.377647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e4257477-f858-49c9-84d1-7d1f50a6f1c4 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:29:59.378850 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e4257477-f858-49c9-84d1-7d1f50a6f1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:59.472775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:29:59.485528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 670/1342] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:29:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:29:59.524023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ed7cc-43c9-4584-96c9-76fdfebfae0f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:29:59.526187 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a10ed7cc-43c9-4584-96c9-76fdfebfae0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:29:59.618760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ed7cc-43c9-4584-96c9-76fdfebfae0f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:29:59.622160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b506293-5ed5-49bd-822c-74c3e5f532b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:29:59.622792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b506293-5ed5-49bd-822c-74c3e5f532b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:29:59.623069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b506293-5ed5-49bd-822c-74c3e5f532b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:29:59.623349 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b506293-5ed5-49bd-822c-74c3e5f532b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:29:59.633265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b506293-5ed5-49bd-822c-74c3e5f532b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:29:59.634689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b506293-5ed5-49bd-822c-74c3e5f532b1 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:29:59.643867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 673/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:29:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:02.857709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefd184-d444-4ced-a3dc-cdf22c56c69c None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:02.858280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefd184-d444-4ced-a3dc-cdf22c56c69c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:02.858280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefd184-d444-4ced-a3dc-cdf22c56c69c None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:02.859641 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 671/1344] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:30:02.864910 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e8859-131a-47f5-8143-a46633fc4669 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:02.865403 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e8859-131a-47f5-8143-a46633fc4669 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:02.865496 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e8859-131a-47f5-8143-a46633fc4669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:02.894174 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-6f4e8859-131a-47f5-8143-a46633fc4669 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:02.895357 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6f4e8859-131a-47f5-8143-a46633fc4669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:02.923714 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:30:02.972723 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 674/1345] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:03.009619 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cefd184-d444-4ced-a3dc-cdf22c56c69c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:03.020308 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cefd184-d444-4ced-a3dc-cdf22c56c69c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:03.023781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30a514-6ab1-4a4b-8a46-7137102036cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:03.024411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30a514-6ab1-4a4b-8a46-7137102036cd service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:03.024686 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30a514-6ab1-4a4b-8a46-7137102036cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:03.024957 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30a514-6ab1-4a4b-8a46-7137102036cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:03.034394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30a514-6ab1-4a4b-8a46-7137102036cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:03.038128 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30a514-6ab1-4a4b-8a46-7137102036cd service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:03.046142 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 672/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:07.449067 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db0df8-4f5c-49c3-8fb2-37301700f514 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:07.449623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db0df8-4f5c-49c3-8fb2-37301700f514 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:07.449623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db0df8-4f5c-49c3-8fb2-37301700f514 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:07.450968 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 675/1347] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:07.456299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac95ad-ca59-468f-8264-7b96a2f49e34 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:07.456757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac95ad-ca59-468f-8264-7b96a2f49e34 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:07.456840 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac95ad-ca59-468f-8264-7b96a2f49e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:07.475074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-02ac95ad-ca59-468f-8264-7b96a2f49e34 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:07.476266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-02ac95ad-ca59-468f-8264-7b96a2f49e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:07.484773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 673/1348] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:07.522187 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db0df8-4f5c-49c3-8fb2-37301700f514 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:07.523887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-86db0df8-4f5c-49c3-8fb2-37301700f514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:07.537248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db0df8-4f5c-49c3-8fb2-37301700f514 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:07.540683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f18cfc-62e4-43b9-8fac-6eee3d572176 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:07.541602 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f18cfc-62e4-43b9-8fac-6eee3d572176 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:07.541884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f18cfc-62e4-43b9-8fac-6eee3d572176 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:07.542157 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f18cfc-62e4-43b9-8fac-6eee3d572176 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:07.551717 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f18cfc-62e4-43b9-8fac-6eee3d572176 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:07.553170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f18cfc-62e4-43b9-8fac-6eee3d572176 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:07.562822 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 676/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:13.332137 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848fca4-ba52-4dd2-8a50-abf357b2676d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:13.333044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848fca4-ba52-4dd2-8a50-abf357b2676d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:13.333567 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848fca4-ba52-4dd2-8a50-abf357b2676d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:13.335876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 674/1350] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jun 7 08:30:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:13.346792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cf08f-dcfa-4114-9d71-830a38e740fa None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:13.346792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cf08f-dcfa-4114-9d71-830a38e740fa None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:13.346792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cf08f-dcfa-4114-9d71-830a38e740fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:13.462249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d73cf08f-dcfa-4114-9d71-830a38e740fa None None] MFA Rules not processed for user `1fc0d7bd7e4547f4a8d90ffc2c2e1d03`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:13.468562 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d73cf08f-dcfa-4114-9d71-830a38e740fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:13.591077 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 677/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jun 7 08:30:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:17.474579 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=13 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:30:18.531205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4bc3d-f6c8-44b0-8ab6-70752e817a3e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:18.532236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4bc3d-f6c8-44b0-8ab6-70752e817a3e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:18.532589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4bc3d-f6c8-44b0-8ab6-70752e817a3e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:18.534062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 675/1352] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jun 7 08:30:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:18.539070 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579880c-7d40-4524-91d7-da3df022b9ff None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:18.539529 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579880c-7d40-4524-91d7-da3df022b9ff None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:18.539615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579880c-7d40-4524-91d7-da3df022b9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:18.569778 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-4579880c-7d40-4524-91d7-da3df022b9ff None None] MFA Rules not processed for user `1fc0d7bd7e4547f4a8d90ffc2c2e1d03`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:18.571487 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4579880c-7d40-4524-91d7-da3df022b9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:18.582657 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 678/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jun 7 08:30:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:19.458917 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3101f-f95a-4a80-91a0-8372a429c4da None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:19.459205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3101f-f95a-4a80-91a0-8372a429c4da None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:19.459477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3101f-f95a-4a80-91a0-8372a429c4da None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:19.460824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 676/1354] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jun 7 08:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:19.465877 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c48a84-39f1-44db-bbf0-2a1f77edfe54 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:19.466236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c48a84-39f1-44db-bbf0-2a1f77edfe54 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:19.466541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c48a84-39f1-44db-bbf0-2a1f77edfe54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:19.489434 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-78c48a84-39f1-44db-bbf0-2a1f77edfe54 None None] MFA Rules not processed for user `1fc0d7bd7e4547f4a8d90ffc2c2e1d03`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:19.490626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-78c48a84-39f1-44db-bbf0-2a1f77edfe54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:19.499240 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 679/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jun 7 08:30:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:23.160081 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cab7a9-8c4b-43e4-98cf-251659ff4573 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:23.160591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cab7a9-8c4b-43e4-98cf-251659ff4573 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:23.160785 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cab7a9-8c4b-43e4-98cf-251659ff4573 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:23.162288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 677/1356] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:23.167645 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29362b0-e15f-4e88-9ea9-f7d719c236f8 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:23.168035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29362b0-e15f-4e88-9ea9-f7d719c236f8 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:23.168198 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29362b0-e15f-4e88-9ea9-f7d719c236f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:23.186906 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b29362b0-e15f-4e88-9ea9-f7d719c236f8 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:23.188949 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b29362b0-e15f-4e88-9ea9-f7d719c236f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:23.200943 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 680/1357] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:23.251713 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe03d7-907d-4b6d-9c72-15704339fa35 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:23.252259 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe03d7-907d-4b6d-9c72-15704339fa35 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:23.252848 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe03d7-907d-4b6d-9c72-15704339fa35 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:23.254515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 678/1358] 10.222.0.17 () {54 vars in 961 bytes} [Tue Jun 7 08:30:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:30:23.264222 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77576e49-574b-4642-abc3-b341928a447a None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:23.264512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77576e49-574b-4642-abc3-b341928a447a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:23.264793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77576e49-574b-4642-abc3-b341928a447a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:23.310128 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:30:23.319066 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-77576e49-574b-4642-abc3-b341928a447a None None] MFA Rules not processed for user `ee42568b059b49a198ed2c01d5efdafd`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:23.320304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-77576e49-574b-4642-abc3-b341928a447a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:23.405952 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 681/1359] 10.222.0.17 () {58 vars in 1058 bytes} [Tue Jun 7 08:30:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:23.417324 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe03d7-907d-4b6d-9c72-15704339fa35 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:23.432319 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe03d7-907d-4b6d-9c72-15704339fa35 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:23.437918 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3cb833-1b61-4dc5-a078-8d7cc39c417e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:23.438559 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cb833-1b61-4dc5-a078-8d7cc39c417e service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:23.438841 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cb833-1b61-4dc5-a078-8d7cc39c417e service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:23.439115 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cb833-1b61-4dc5-a078-8d7cc39c417e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:23.447874 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3cb833-1b61-4dc5-a078-8d7cc39c417e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:23.449481 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3cb833-1b61-4dc5-a078-8d7cc39c417e service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:23.459686 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 679/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:25.924790 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564db42-59d9-41fa-8c23-3b7159d66ec7 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:25.925352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564db42-59d9-41fa-8c23-3b7159d66ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:25.925471 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564db42-59d9-41fa-8c23-3b7159d66ec7 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:25.926973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 682/1361] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:25.932020 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99991f-becc-4c46-b6b3-d5f4ab2a96e1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:25.932423 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99991f-becc-4c46-b6b3-d5f4ab2a96e1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:25.932579 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99991f-becc-4c46-b6b3-d5f4ab2a96e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:25.951126 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bf99991f-becc-4c46-b6b3-d5f4ab2a96e1 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:25.952315 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bf99991f-becc-4c46-b6b3-d5f4ab2a96e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:25.960494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 680/1362] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:26.005363 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564db42-59d9-41fa-8c23-3b7159d66ec7 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:26.019507 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564db42-59d9-41fa-8c23-3b7159d66ec7 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:26.024810 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce0e8d-2b99-4151-a22c-5e4ee099ad47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:26.025497 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce0e8d-2b99-4151-a22c-5e4ee099ad47 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:26.025780 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce0e8d-2b99-4151-a22c-5e4ee099ad47 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:26.026062 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce0e8d-2b99-4151-a22c-5e4ee099ad47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:26.034640 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce0e8d-2b99-4151-a22c-5e4ee099ad47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:26.036220 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce0e8d-2b99-4151-a22c-5e4ee099ad47 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:26.046688 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 683/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:28.675765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36e0ec-18d6-47d0-9cb0-9d66dbe791cb None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:28.676297 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36e0ec-18d6-47d0-9cb0-9d66dbe791cb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:28.676297 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36e0ec-18d6-47d0-9cb0-9d66dbe791cb None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:28.677773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 681/1364] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:28.683050 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ce8ac-1067-4053-8c18-b11dccfddd94 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:28.683494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ce8ac-1067-4053-8c18-b11dccfddd94 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:28.683588 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ce8ac-1067-4053-8c18-b11dccfddd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:28.702173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d25ce8ac-1067-4053-8c18-b11dccfddd94 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:28.703347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d25ce8ac-1067-4053-8c18-b11dccfddd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:28.712528 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 684/1365] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:28.769951 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af90e6-3a59-4636-a1f2-99be37e4b0ba None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:28.781162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af90e6-3a59-4636-a1f2-99be37e4b0ba None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:28.781162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af90e6-3a59-4636-a1f2-99be37e4b0ba None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:28.781162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 682/1366] 10.222.0.17 () {54 vars in 961 bytes} [Tue Jun 7 08:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:30:28.781734 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3396de9-7e47-4dd9-9f27-63305f68846b None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:28.781734 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3396de9-7e47-4dd9-9f27-63305f68846b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:28.781734 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3396de9-7e47-4dd9-9f27-63305f68846b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:28.797888 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d3396de9-7e47-4dd9-9f27-63305f68846b None None] MFA Rules not processed for user `ee42568b059b49a198ed2c01d5efdafd`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:28.799072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d3396de9-7e47-4dd9-9f27-63305f68846b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:28.808764 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 685/1367] 10.222.0.17 () {58 vars in 1058 bytes} [Tue Jun 7 08:30:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:30:28.817045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af90e6-3a59-4636-a1f2-99be37e4b0ba None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:28.831319 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af90e6-3a59-4636-a1f2-99be37e4b0ba None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:28.836772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b67b2a-cfe9-4411-8fbf-93a3a7b7621e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:28.837532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b67b2a-cfe9-4411-8fbf-93a3a7b7621e service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:28.837816 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b67b2a-cfe9-4411-8fbf-93a3a7b7621e service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:28.838099 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b67b2a-cfe9-4411-8fbf-93a3a7b7621e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:28.846292 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b67b2a-cfe9-4411-8fbf-93a3a7b7621e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:28.847999 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b67b2a-cfe9-4411-8fbf-93a3a7b7621e service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:28.858047 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 683/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:31.497139 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c6477-c578-41ef-9f9b-6b9046e05f64 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:31.498668 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c6477-c578-41ef-9f9b-6b9046e05f64 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:31.499189 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c6477-c578-41ef-9f9b-6b9046e05f64 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:31.504957 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 686/1369] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:30:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:31.506389 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54842053-07d6-4a06-863f-75c543bc326d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:31.507373 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54842053-07d6-4a06-863f-75c543bc326d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:31.507813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54842053-07d6-4a06-863f-75c543bc326d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:31.535447 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-54842053-07d6-4a06-863f-75c543bc326d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:31.536901 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-54842053-07d6-4a06-863f-75c543bc326d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:31.546826 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 684/1370] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:30:31.594726 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c6477-c578-41ef-9f9b-6b9046e05f64 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:31.610990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c6477-c578-41ef-9f9b-6b9046e05f64 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:31.617529 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a404a9-c34f-4b4f-9338-6aa6f63ecfd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:31.618803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a404a9-c34f-4b4f-9338-6aa6f63ecfd9 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:31.619325 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a404a9-c34f-4b4f-9338-6aa6f63ecfd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:31.619819 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a404a9-c34f-4b4f-9338-6aa6f63ecfd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:31.630615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a404a9-c34f-4b4f-9338-6aa6f63ecfd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:31.632924 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a404a9-c34f-4b4f-9338-6aa6f63ecfd9 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:31.654419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 687/1371] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:31.656286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b6283-f2a0-4838-b135-d4a53cbd92de None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:31.656633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b6283-f2a0-4838-b135-d4a53cbd92de None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:31.657029 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b6283-f2a0-4838-b135-d4a53cbd92de None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:31.658459 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 685/1372] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:31.664162 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e67fb3-db25-484b-8804-b320903106d7 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:31.664512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e67fb3-db25-484b-8804-b320903106d7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:31.664760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e67fb3-db25-484b-8804-b320903106d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:31.683966 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-62e67fb3-db25-484b-8804-b320903106d7 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:31.685756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-62e67fb3-db25-484b-8804-b320903106d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:31.694038 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 688/1373] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:32.038674 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252a92c-78e9-4597-b19a-6029ee307b7d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:32.038960 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252a92c-78e9-4597-b19a-6029ee307b7d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:32.039404 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252a92c-78e9-4597-b19a-6029ee307b7d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:32.041758 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 686/1374] 10.222.0.17 () {54 vars in 963 bytes} [Tue Jun 7 08:30:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:32.051041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7555a-8cbf-4ea6-a046-30057e2ce7f9 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:32.051779 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7555a-8cbf-4ea6-a046-30057e2ce7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:32.052664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7555a-8cbf-4ea6-a046-30057e2ce7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:32.122087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-43e7555a-8cbf-4ea6-a046-30057e2ce7f9 None None] MFA Rules not processed for user `85ea2962e7094f8c878d95680764c4e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:32.123552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-43e7555a-8cbf-4ea6-a046-30057e2ce7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:32.204609 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 689/1375] 10.222.0.17 () {58 vars in 1060 bytes} [Tue Jun 7 08:30:32 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:32.212639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252a92c-78e9-4597-b19a-6029ee307b7d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:32.226980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252a92c-78e9-4597-b19a-6029ee307b7d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:32.231169 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8358c-9442-436e-92ae-c14618b2d5bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:32.231779 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8358c-9442-436e-92ae-c14618b2d5bd service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:32.232609 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8358c-9442-436e-92ae-c14618b2d5bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:32.232895 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8358c-9442-436e-92ae-c14618b2d5bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:32.250200 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8358c-9442-436e-92ae-c14618b2d5bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:32.251730 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8358c-9442-436e-92ae-c14618b2d5bd service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:32.263963 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 687/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:34.138126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1638d-36fb-414a-9b4a-744f50c4c0cd None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:34.139198 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1638d-36fb-414a-9b4a-744f50c4c0cd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:34.139294 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1638d-36fb-414a-9b4a-744f50c4c0cd None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:34.140649 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 690/1377] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:30:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:34.146083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9191cb-2e18-493f-a700-ba36960437e9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:34.146528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9191cb-2e18-493f-a700-ba36960437e9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:34.146628 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9191cb-2e18-493f-a700-ba36960437e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:34.169043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ab9191cb-2e18-493f-a700-ba36960437e9 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:34.170240 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ab9191cb-2e18-493f-a700-ba36960437e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:34.178948 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 688/1378] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:34.225638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1638d-36fb-414a-9b4a-744f50c4c0cd None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:34.246841 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1638d-36fb-414a-9b4a-744f50c4c0cd None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:34.253835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e6243-cc31-4098-b9b4-395f9e9d25c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:34.257896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e6243-cc31-4098-b9b4-395f9e9d25c4 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:34.258183 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e6243-cc31-4098-b9b4-395f9e9d25c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:34.258799 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e6243-cc31-4098-b9b4-395f9e9d25c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:34.270068 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e6243-cc31-4098-b9b4-395f9e9d25c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:34.272047 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e6243-cc31-4098-b9b4-395f9e9d25c4 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:34.279759 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:30:34.285171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 691/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:36.794358 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42603345-7a04-4404-abdf-805a59d0f0b2 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:36.804836 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42603345-7a04-4404-abdf-805a59d0f0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:36.804836 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42603345-7a04-4404-abdf-805a59d0f0b2 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:36.804836 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 689/1380] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:30:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:36.805176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749299e8-bf42-4328-b554-9092a22ed300 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:36.805176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749299e8-bf42-4328-b554-9092a22ed300 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:36.805176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749299e8-bf42-4328-b554-9092a22ed300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:36.825097 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-749299e8-bf42-4328-b554-9092a22ed300 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:36.826633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-749299e8-bf42-4328-b554-9092a22ed300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:36.834985 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 692/1381] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:36.881804 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42603345-7a04-4404-abdf-805a59d0f0b2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:36.898458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42603345-7a04-4404-abdf-805a59d0f0b2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:36.905554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d850162-95fd-4673-ae7e-ed0b0850e719 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:36.906726 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d850162-95fd-4673-ae7e-ed0b0850e719 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:36.907211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d850162-95fd-4673-ae7e-ed0b0850e719 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:36.907578 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d850162-95fd-4673-ae7e-ed0b0850e719 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:36.918015 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d850162-95fd-4673-ae7e-ed0b0850e719 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:36.920746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d850162-95fd-4673-ae7e-ed0b0850e719 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:36.939329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 690/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:39.350369 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b6c68-2023-47a9-a2d6-40f2e8c7dd54 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:39.350934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b6c68-2023-47a9-a2d6-40f2e8c7dd54 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:39.350934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b6c68-2023-47a9-a2d6-40f2e8c7dd54 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:39.352409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 693/1383] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:39.359167 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdf24e-4293-49ec-8d5b-9070938ccf11 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:39.359755 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdf24e-4293-49ec-8d5b-9070938ccf11 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:39.359755 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdf24e-4293-49ec-8d5b-9070938ccf11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:39.383438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-dabdf24e-4293-49ec-8d5b-9070938ccf11 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:39.385360 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dabdf24e-4293-49ec-8d5b-9070938ccf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:39.393927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 691/1384] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:39.442219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b6c68-2023-47a9-a2d6-40f2e8c7dd54 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:39.452206 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b6c68-2023-47a9-a2d6-40f2e8c7dd54 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:39.457736 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e48bd-8f15-4be1-b8a7-2b157414c778 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:39.458360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e48bd-8f15-4be1-b8a7-2b157414c778 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:39.458637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e48bd-8f15-4be1-b8a7-2b157414c778 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:39.458909 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e48bd-8f15-4be1-b8a7-2b157414c778 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:39.467852 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e48bd-8f15-4be1-b8a7-2b157414c778 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:39.469469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e48bd-8f15-4be1-b8a7-2b157414c778 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:39.479093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 694/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:41.746539 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd100a-88a4-4c69-b1f4-a6fc29490bad None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:41.747253 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd100a-88a4-4c69-b1f4-a6fc29490bad None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:41.747253 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd100a-88a4-4c69-b1f4-a6fc29490bad None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:41.749122 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 692/1386] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:30:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:41.754333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26db83a-7c33-4b33-8f0d-d681dbf2dbaf None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:41.754827 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26db83a-7c33-4b33-8f0d-d681dbf2dbaf None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:41.754993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26db83a-7c33-4b33-8f0d-d681dbf2dbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:41.777987 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e26db83a-7c33-4b33-8f0d-d681dbf2dbaf None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:41.779666 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e26db83a-7c33-4b33-8f0d-d681dbf2dbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:41.790544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 695/1387] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:41.849479 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd100a-88a4-4c69-b1f4-a6fc29490bad None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:41.867677 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd100a-88a4-4c69-b1f4-a6fc29490bad None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:41.874909 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e3541-cc29-471c-9264-fe0221ebeda4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:41.875806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e3541-cc29-471c-9264-fe0221ebeda4 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:41.876215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e3541-cc29-471c-9264-fe0221ebeda4 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:41.876643 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e3541-cc29-471c-9264-fe0221ebeda4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:41.887168 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e3541-cc29-471c-9264-fe0221ebeda4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:41.889683 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e3541-cc29-471c-9264-fe0221ebeda4 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:41.903550 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 693/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:44.308373 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4e576-4652-4b92-b686-80934c3333c4 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:44.309101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4e576-4652-4b92-b686-80934c3333c4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:44.309101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4e576-4652-4b92-b686-80934c3333c4 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:44.310385 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 696/1389] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:44.315841 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a313f1c-f0f2-4487-934b-ed591b6c7c03 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:44.316411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a313f1c-f0f2-4487-934b-ed591b6c7c03 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:44.316411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a313f1c-f0f2-4487-934b-ed591b6c7c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:44.334812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8a313f1c-f0f2-4487-934b-ed591b6c7c03 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:44.336024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8a313f1c-f0f2-4487-934b-ed591b6c7c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:44.344541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 694/1390] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:44.400736 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4e576-4652-4b92-b686-80934c3333c4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:44.420587 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4e576-4652-4b92-b686-80934c3333c4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:44.427826 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e927d9e-9f11-4406-9c98-914c39a5a84d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:44.428694 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e927d9e-9f11-4406-9c98-914c39a5a84d service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:44.429097 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e927d9e-9f11-4406-9c98-914c39a5a84d service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:44.429449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e927d9e-9f11-4406-9c98-914c39a5a84d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:44.440780 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e927d9e-9f11-4406-9c98-914c39a5a84d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:44.442985 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e927d9e-9f11-4406-9c98-914c39a5a84d service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:44.454915 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 697/1391] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:47.291785 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e4adc-ea2c-4a64-b09f-920d606c12b5 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:47.292386 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e4adc-ea2c-4a64-b09f-920d606c12b5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:47.292386 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e4adc-ea2c-4a64-b09f-920d606c12b5 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:47.293720 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 695/1392] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:47.299006 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dceb6a-e0ad-44a2-b03c-f256fc7d5d07 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:47.299453 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dceb6a-e0ad-44a2-b03c-f256fc7d5d07 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:47.299559 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dceb6a-e0ad-44a2-b03c-f256fc7d5d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:47.317420 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-37dceb6a-e0ad-44a2-b03c-f256fc7d5d07 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:47.318622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-37dceb6a-e0ad-44a2-b03c-f256fc7d5d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:47.326917 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 698/1393] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:30:47.369980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e4adc-ea2c-4a64-b09f-920d606c12b5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:47.384766 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e4adc-ea2c-4a64-b09f-920d606c12b5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:47.390806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9193c-9a31-4da4-8b4c-8f394a5df8cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:47.391445 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9193c-9a31-4da4-8b4c-8f394a5df8cd service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:47.391729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9193c-9a31-4da4-8b4c-8f394a5df8cd service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:47.392008 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9193c-9a31-4da4-8b4c-8f394a5df8cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:47.399980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9193c-9a31-4da4-8b4c-8f394a5df8cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:47.401665 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9193c-9a31-4da4-8b4c-8f394a5df8cd service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:47.410892 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 696/1394] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:48.751079 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71f747-0251-452d-93d1-29e2d700cf25 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:48.751686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71f747-0251-452d-93d1-29e2d700cf25 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:48.751686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71f747-0251-452d-93d1-29e2d700cf25 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:48.753034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 699/1395] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:48.758378 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa053e2-0477-485c-ba28-6d8b3521cfdb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:48.758815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa053e2-0477-485c-ba28-6d8b3521cfdb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:48.758922 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa053e2-0477-485c-ba28-6d8b3521cfdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:48.765136 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=52 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:30:48.777057 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2aa053e2-0477-485c-ba28-6d8b3521cfdb None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:48.783237 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2aa053e2-0477-485c-ba28-6d8b3521cfdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:48.786613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 697/1396] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:48.842489 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f4278-83a4-47fb-8f99-7b1d883f3e38 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:48.842772 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f4278-83a4-47fb-8f99-7b1d883f3e38 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:48.843047 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f4278-83a4-47fb-8f99-7b1d883f3e38 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:48.844427 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 700/1397] 10.222.0.17 () {54 vars in 962 bytes} [Tue Jun 7 08:30:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:30:48.849792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cff41-022d-48d5-bf1a-4d653c3a4168 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:48.850087 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cff41-022d-48d5-bf1a-4d653c3a4168 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:48.850369 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cff41-022d-48d5-bf1a-4d653c3a4168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:48.897444 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-147cff41-022d-48d5-bf1a-4d653c3a4168 None None] MFA Rules not processed for user `c4d9b5e3826049f2800f1b9424977d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:48.898643 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-147cff41-022d-48d5-bf1a-4d653c3a4168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:48.963790 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 698/1398] 10.222.0.17 () {58 vars in 1059 bytes} [Tue Jun 7 08:30:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:48.972092 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f4278-83a4-47fb-8f99-7b1d883f3e38 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:48.987187 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f4278-83a4-47fb-8f99-7b1d883f3e38 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:48.991928 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942db3f0-9474-4d4b-a776-ff2b67062e59 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4d9b5e3826049f2800f1b9424977d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:48.992582 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942db3f0-9474-4d4b-a776-ff2b67062e59 service glance] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:48.992857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942db3f0-9474-4d4b-a776-ff2b67062e59 service glance] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:48.993166 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942db3f0-9474-4d4b-a776-ff2b67062e59 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:49.002231 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942db3f0-9474-4d4b-a776-ff2b67062e59 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:49.003620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942db3f0-9474-4d4b-a776-ff2b67062e59 service glance] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:49.013449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 701/1399] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:30:49.690489 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6191a6-1f4d-4906-b88d-7906694a67db None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:49.690872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6191a6-1f4d-4906-b88d-7906694a67db None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:49.691214 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6191a6-1f4d-4906-b88d-7906694a67db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:49.743360 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-cb6191a6-1f4d-4906-b88d-7906694a67db None None] MFA Rules not processed for user `bbf0ce396885472bb5d225aa009043f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:49.744561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cb6191a6-1f4d-4906-b88d-7906694a67db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:49.788834 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc4fd5-5eff-4ce6-8931-abb56c3d1578 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:49.789423 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc4fd5-5eff-4ce6-8931-abb56c3d1578 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:49.789423 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc4fd5-5eff-4ce6-8931-abb56c3d1578 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:49.791070 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 702/1400] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:30:49.796626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7c07e-d2c3-43e1-ae90-77b6e76b933a None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:49.797067 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7c07e-d2c3-43e1-ae90-77b6e76b933a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:49.797351 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7c07e-d2c3-43e1-ae90-77b6e76b933a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:49.809389 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 699/1401] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jun 7 08:30:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:49.815223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ebe7c07e-d2c3-43e1-ae90-77b6e76b933a None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:49.816412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ebe7c07e-d2c3-43e1-ae90-77b6e76b933a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:49.825279 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 703/1402] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:30:49.851433 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ec9c8-1caa-41f5-9c8c-08fd36572b5d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:49.851732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ec9c8-1caa-41f5-9c8c-08fd36572b5d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:49.852069 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ec9c8-1caa-41f5-9c8c-08fd36572b5d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:49.853558 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 700/1403] 10.222.0.17 () {54 vars in 942 bytes} [Tue Jun 7 08:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:49.860708 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b5ad6-3204-44fe-a0fa-e4602a8399ff None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:49.861016 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b5ad6-3204-44fe-a0fa-e4602a8399ff None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:49.861377 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b5ad6-3204-44fe-a0fa-e4602a8399ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:49.869184 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ec9c8-1caa-41f5-9c8c-08fd36572b5d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:49.884259 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ec9c8-1caa-41f5-9c8c-08fd36572b5d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:49.889797 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd274f6-c8c5-4cc0-bede-4a280c687534 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:49.890560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd274f6-c8c5-4cc0-bede-4a280c687534 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:49.890851 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd274f6-c8c5-4cc0-bede-4a280c687534 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:49.891125 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd274f6-c8c5-4cc0-bede-4a280c687534 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:49.899541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd274f6-c8c5-4cc0-bede-4a280c687534 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:49.901168 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd274f6-c8c5-4cc0-bede-4a280c687534 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:49.910074 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a57b5ad6-3204-44fe-a0fa-e4602a8399ff None None] MFA Rules not processed for user `334dc47ae2e643ccb44d25c6b5906f14`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:49.910822 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 701/1404] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:30:49.911273 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a57b5ad6-3204-44fe-a0fa-e4602a8399ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:49.989166 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 704/1405] 10.222.0.17 () {58 vars in 1039 bytes} [Tue Jun 7 08:30:49 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:30:49.993197 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd274f6-c8c5-4cc0-bede-4a280c687534 service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:50.005057 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd274f6-c8c5-4cc0-bede-4a280c687534 service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:50.010114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b93d0-c247-47c9-8498-96b52dff4794 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334dc47ae2e643ccb44d25c6b5906f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:50.010747 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b93d0-c247-47c9-8498-96b52dff4794 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:50.011029 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b93d0-c247-47c9-8498-96b52dff4794 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:50.011307 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b93d0-c247-47c9-8498-96b52dff4794 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:50.019623 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b93d0-c247-47c9-8498-96b52dff4794 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:50.023533 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b93d0-c247-47c9-8498-96b52dff4794 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:50.029340 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 702/1406] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jun 7 08:30:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:30:52.369970 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36eafc-f5d1-4526-a63a-bd3c9c3b418d None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:52.473649 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36eafc-f5d1-4526-a63a-bd3c9c3b418d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:52.473649 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36eafc-f5d1-4526-a63a-bd3c9c3b418d None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:52.473649 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 705/1407] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:30:52.475004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0e0f1-7ec6-4b24-a4df-52028a60d702 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:52.475004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0e0f1-7ec6-4b24-a4df-52028a60d702 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:52.475004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0e0f1-7ec6-4b24-a4df-52028a60d702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:52.475004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-abf0e0f1-7ec6-4b24-a4df-52028a60d702 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:30:52.475004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-abf0e0f1-7ec6-4b24-a4df-52028a60d702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:30:52.475004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 703/1408] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:30:52.480936 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36eafc-f5d1-4526-a63a-bd3c9c3b418d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:30:52.516470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36eafc-f5d1-4526-a63a-bd3c9c3b418d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:30:52.524576 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923a44f-0202-44b6-89f6-04391ab668d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:30:52.525522 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923a44f-0202-44b6-89f6-04391ab668d3 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:30:52.525805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923a44f-0202-44b6-89f6-04391ab668d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:30:52.526091 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923a44f-0202-44b6-89f6-04391ab668d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:30:52.536433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923a44f-0202-44b6-89f6-04391ab668d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:30:52.538074 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923a44f-0202-44b6-89f6-04391ab668d3 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:30:52.549860 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 706/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:30:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:31:02.397865 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:31:02.636568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=81 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:31:15.105037 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af8be9-cbcf-4d8b-8a0f-307ad1e143d0 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:15.105672 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af8be9-cbcf-4d8b-8a0f-307ad1e143d0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:15.105672 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af8be9-cbcf-4d8b-8a0f-307ad1e143d0 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:15.106971 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 704/1410] 10.222.0.17 () {54 vars in 918 bytes} [Tue Jun 7 08:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:15.112553 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270aa63-3ff3-4bbd-bf49-5bb2eb80102e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:15.113067 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270aa63-3ff3-4bbd-bf49-5bb2eb80102e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:15.113176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270aa63-3ff3-4bbd-bf49-5bb2eb80102e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:15.130726 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e270aa63-3ff3-4bbd-bf49-5bb2eb80102e None None] MFA Rules not processed for user `1fc0d7bd7e4547f4a8d90ffc2c2e1d03`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:15.131924 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e270aa63-3ff3-4bbd-bf49-5bb2eb80102e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:15.140264 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 707/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Tue Jun 7 08:31:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:31:15.157225 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af876766-4e76-4db9-919e-79c23d3a623b None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:15.157520 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af876766-4e76-4db9-919e-79c23d3a623b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:15.157801 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af876766-4e76-4db9-919e-79c23d3a623b None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:15.159176 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 705/1412] 10.222.0.17 () {54 vars in 918 bytes} [Tue Jun 7 08:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:15.164567 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6189b95-7de8-4b6b-8792-19920baf6ff4 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:15.164883 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6189b95-7de8-4b6b-8792-19920baf6ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:15.165454 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6189b95-7de8-4b6b-8792-19920baf6ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:15.184864 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c6189b95-7de8-4b6b-8792-19920baf6ff4 None None] MFA Rules not processed for user `ee42568b059b49a198ed2c01d5efdafd`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:15.186042 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c6189b95-7de8-4b6b-8792-19920baf6ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:15.201789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 708/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Tue Jun 7 08:31:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:31:17.976075 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffd198-865b-4824-b123-340c8fd2488f None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:17.976621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffd198-865b-4824-b123-340c8fd2488f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:17.976621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffd198-865b-4824-b123-340c8fd2488f None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:17.978024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:17.983241 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a98c8-dae0-4f95-b485-2205c525ed5c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:17.983693 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a98c8-dae0-4f95-b485-2205c525ed5c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:17.983868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a98c8-dae0-4f95-b485-2205c525ed5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:18.002547 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-896a98c8-dae0-4f95-b485-2205c525ed5c None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:18.003722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-896a98c8-dae0-4f95-b485-2205c525ed5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:18.011678 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:31:18.054506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffd198-865b-4824-b123-340c8fd2488f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:31:18.068534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffd198-865b-4824-b123-340c8fd2488f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:31:18.074190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eafc10-1dd0-4e75-a9b1-04e8a1397499 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:31:18.074537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eafc10-1dd0-4e75-a9b1-04e8a1397499 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:18.074815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eafc10-1dd0-4e75-a9b1-04e8a1397499 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:18.075084 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eafc10-1dd0-4e75-a9b1-04e8a1397499 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:18.083118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eafc10-1dd0-4e75-a9b1-04e8a1397499 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:31:18.084731 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eafc10-1dd0-4e75-a9b1-04e8a1397499 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:31:18.095000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 707/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:31:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:31:19.087616 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a80757-910e-420b-b759-45ef33bd0017 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:19.088195 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a80757-910e-420b-b759-45ef33bd0017 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:19.088195 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a80757-910e-420b-b759-45ef33bd0017 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:19.089562 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:19.095190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf216b7-264f-4bb7-8362-8062eaebe747 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:19.095660 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf216b7-264f-4bb7-8362-8062eaebe747 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:19.095773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf216b7-264f-4bb7-8362-8062eaebe747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:19.113326 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0bf216b7-264f-4bb7-8362-8062eaebe747 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:19.114509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0bf216b7-264f-4bb7-8362-8062eaebe747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:19.122181 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:31:19.329502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a80757-910e-420b-b759-45ef33bd0017 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:31:19.343521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a80757-910e-420b-b759-45ef33bd0017 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:31:19.348917 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e9f22-a282-4259-aa77-f17b84cbf404 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:31:19.349628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e9f22-a282-4259-aa77-f17b84cbf404 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:19.350005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e9f22-a282-4259-aa77-f17b84cbf404 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:19.350384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e9f22-a282-4259-aa77-f17b84cbf404 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:19.358932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e9f22-a282-4259-aa77-f17b84cbf404 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:31:19.360613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e9f22-a282-4259-aa77-f17b84cbf404 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:31:19.370300 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 711/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:31:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:31:20.343124 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450476d-4da5-480b-b791-c1590ddfdf76 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:20.343686 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450476d-4da5-480b-b791-c1590ddfdf76 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:20.343686 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450476d-4da5-480b-b791-c1590ddfdf76 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:20.345043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:20.350032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242d219-3c34-4f55-9f0f-fe07c0232340 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:20.350452 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242d219-3c34-4f55-9f0f-fe07c0232340 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:20.350612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242d219-3c34-4f55-9f0f-fe07c0232340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:20.368254 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b242d219-3c34-4f55-9f0f-fe07c0232340 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:20.369453 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b242d219-3c34-4f55-9f0f-fe07c0232340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:20.377718 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:31:20.526165 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450476d-4da5-480b-b791-c1590ddfdf76 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:31:20.539963 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450476d-4da5-480b-b791-c1590ddfdf76 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:31:20.545524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ce2a1-cbb4-415f-b353-3d967c9dca1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:31:20.546695 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ce2a1-cbb4-415f-b353-3d967c9dca1d service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:20.546972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ce2a1-cbb4-415f-b353-3d967c9dca1d service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:20.547245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ce2a1-cbb4-415f-b353-3d967c9dca1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:20.555877 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ce2a1-cbb4-415f-b353-3d967c9dca1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:31:20.557498 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ce2a1-cbb4-415f-b353-3d967c9dca1d service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:31:20.567694 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 710/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:31:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:31:21.462656 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58515c-7772-41a3-97df-423e641867f0 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:21.463226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58515c-7772-41a3-97df-423e641867f0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:21.463226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58515c-7772-41a3-97df-423e641867f0 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:21.464554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:21.470129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b415b9-4440-4a1e-95a7-e76c4bb1401b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:21.470561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b415b9-4440-4a1e-95a7-e76c4bb1401b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:21.470685 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b415b9-4440-4a1e-95a7-e76c4bb1401b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:21.489298 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a4b415b9-4440-4a1e-95a7-e76c4bb1401b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:21.490493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a4b415b9-4440-4a1e-95a7-e76c4bb1401b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:21.498138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:31:21.647201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58515c-7772-41a3-97df-423e641867f0 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:31:21.661402 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58515c-7772-41a3-97df-423e641867f0 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:31:21.667418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37611bb-e2e0-47e1-81dc-a9e806c1303d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:31:21.668071 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37611bb-e2e0-47e1-81dc-a9e806c1303d service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:21.668470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37611bb-e2e0-47e1-81dc-a9e806c1303d service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:21.668770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37611bb-e2e0-47e1-81dc-a9e806c1303d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:21.677288 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37611bb-e2e0-47e1-81dc-a9e806c1303d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:31:21.678918 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37611bb-e2e0-47e1-81dc-a9e806c1303d service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:31:21.689088 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 714/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:31:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:31:22.597216 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c9b2a-9f28-44ff-831f-a66670f7cd0b None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:22.597730 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c9b2a-9f28-44ff-831f-a66670f7cd0b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:22.597730 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c9b2a-9f28-44ff-831f-a66670f7cd0b None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:22.599062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:22.604059 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209e9eb-d989-4132-83ae-128fbb04d289 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:22.604486 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209e9eb-d989-4132-83ae-128fbb04d289 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:22.604605 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209e9eb-d989-4132-83ae-128fbb04d289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:22.622454 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1209e9eb-d989-4132-83ae-128fbb04d289 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:22.623661 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1209e9eb-d989-4132-83ae-128fbb04d289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:22.632323 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:31:22.783065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c9b2a-9f28-44ff-831f-a66670f7cd0b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:31:22.798487 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c9b2a-9f28-44ff-831f-a66670f7cd0b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:31:22.803704 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc529e-324c-4c26-80af-2780afd87fbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:31:22.804332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc529e-324c-4c26-80af-2780afd87fbc service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:22.804616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc529e-324c-4c26-80af-2780afd87fbc service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:22.804882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc529e-324c-4c26-80af-2780afd87fbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:22.813614 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc529e-324c-4c26-80af-2780afd87fbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:31:22.815193 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc529e-324c-4c26-80af-2780afd87fbc service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:31:22.824745 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 713/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:31:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:31:23.771754 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8347cab-2b00-495e-8a44-703c11d73913 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:23.772297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8347cab-2b00-495e-8a44-703c11d73913 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:23.772297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8347cab-2b00-495e-8a44-703c11d73913 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:23.773699 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:23.778748 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5981275-7fc2-465a-857e-adbc1e1eda6a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:23.779177 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5981275-7fc2-465a-857e-adbc1e1eda6a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:23.779336 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5981275-7fc2-465a-857e-adbc1e1eda6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:23.797909 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c5981275-7fc2-465a-857e-adbc1e1eda6a None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:23.799112 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c5981275-7fc2-465a-857e-adbc1e1eda6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:23.806921 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:31:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:31:23.956527 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8347cab-2b00-495e-8a44-703c11d73913 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:31:23.971273 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8347cab-2b00-495e-8a44-703c11d73913 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:31:23.976613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8329a-37c9-43a5-9f25-586709745314 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:31:23.977287 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8329a-37c9-43a5-9f25-586709745314 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:23.977569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8329a-37c9-43a5-9f25-586709745314 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:23.977845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8329a-37c9-43a5-9f25-586709745314 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:23.986782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8329a-37c9-43a5-9f25-586709745314 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:31:23.988370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8329a-37c9-43a5-9f25-586709745314 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:31:23.998973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 717/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:31:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:31:24.914455 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3900d1-1440-4968-973e-1dc43e183a6e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:24.915047 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3900d1-1440-4968-973e-1dc43e183a6e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:24.915047 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3900d1-1440-4968-973e-1dc43e183a6e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:24.916621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:24.922225 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defcfc0-8fb3-403b-b499-4f21fa70c4d1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:24.922670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defcfc0-8fb3-403b-b499-4f21fa70c4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:24.928630 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defcfc0-8fb3-403b-b499-4f21fa70c4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:24.941237 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7defcfc0-8fb3-403b-b499-4f21fa70c4d1 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:24.942432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7defcfc0-8fb3-403b-b499-4f21fa70c4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:24.950248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:31:25.100468 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3900d1-1440-4968-973e-1dc43e183a6e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:31:25.115175 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3900d1-1440-4968-973e-1dc43e183a6e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:31:25.120571 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f138217-01af-4979-bc0e-7c9e4edd7a50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:31:25.121241 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f138217-01af-4979-bc0e-7c9e4edd7a50 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:25.121524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f138217-01af-4979-bc0e-7c9e4edd7a50 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:25.121800 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f138217-01af-4979-bc0e-7c9e4edd7a50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:25.130679 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f138217-01af-4979-bc0e-7c9e4edd7a50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:31:25.132261 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f138217-01af-4979-bc0e-7c9e4edd7a50 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:31:25.141831 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 716/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:31:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:31:26.054783 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce1c5d-f2ee-40f8-a00f-f46c9405bf96 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:26.055358 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce1c5d-f2ee-40f8-a00f-f46c9405bf96 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:26.055358 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce1c5d-f2ee-40f8-a00f-f46c9405bf96 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:26.056690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:26.062411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75fb98-f39f-46fc-9350-3ba8968b7c5b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:26.062843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75fb98-f39f-46fc-9350-3ba8968b7c5b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:26.062958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75fb98-f39f-46fc-9350-3ba8968b7c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:26.081369 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1c75fb98-f39f-46fc-9350-3ba8968b7c5b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:26.082572 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1c75fb98-f39f-46fc-9350-3ba8968b7c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:26.090257 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:31:26.240002 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce1c5d-f2ee-40f8-a00f-f46c9405bf96 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:31:26.254742 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce1c5d-f2ee-40f8-a00f-f46c9405bf96 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:31:26.259995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce64cb-238d-483f-9752-f43a818a707a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:31:26.260625 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce64cb-238d-483f-9752-f43a818a707a service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:26.260910 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce64cb-238d-483f-9752-f43a818a707a service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:26.261213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce64cb-238d-483f-9752-f43a818a707a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:26.270142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce64cb-238d-483f-9752-f43a818a707a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:31:26.271736 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce64cb-238d-483f-9752-f43a818a707a service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:31:26.281240 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 720/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:31:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:31:27.180981 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e4e5f-e06f-46bd-9126-029e65513320 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:27.181545 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e4e5f-e06f-46bd-9126-029e65513320 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:27.181545 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e4e5f-e06f-46bd-9126-029e65513320 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:27.182962 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:27.188193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e318e-e8b8-45c7-86ef-bf2edd89146a None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:27.188580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e318e-e8b8-45c7-86ef-bf2edd89146a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:27.188734 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e318e-e8b8-45c7-86ef-bf2edd89146a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:27.207073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ce8e318e-e8b8-45c7-86ef-bf2edd89146a None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:27.208255 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ce8e318e-e8b8-45c7-86ef-bf2edd89146a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:27.216025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:31:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:31:27.364763 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e4e5f-e06f-46bd-9126-029e65513320 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:31:27.378979 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e4e5f-e06f-46bd-9126-029e65513320 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:31:27.384469 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0329a-9b5c-44e9-a9bf-1d637c16b198 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:31:27.385144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0329a-9b5c-44e9-a9bf-1d637c16b198 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:27.385428 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0329a-9b5c-44e9-a9bf-1d637c16b198 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:27.385709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0329a-9b5c-44e9-a9bf-1d637c16b198 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:27.394721 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0329a-9b5c-44e9-a9bf-1d637c16b198 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:31:27.396340 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0329a-9b5c-44e9-a9bf-1d637c16b198 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:31:27.406705 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 719/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:31:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:31:29.517662 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e184b2d-ad39-44da-8ccc-5344c54249cd None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:29.518257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e184b2d-ad39-44da-8ccc-5344c54249cd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:29.518257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e184b2d-ad39-44da-8ccc-5344c54249cd None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:29.519625 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:29.525288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412e951-b6a3-4f65-9746-73ef08879184 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:29.525740 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412e951-b6a3-4f65-9746-73ef08879184 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:29.525842 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412e951-b6a3-4f65-9746-73ef08879184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:29.544236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2412e951-b6a3-4f65-9746-73ef08879184 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:29.545460 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2412e951-b6a3-4f65-9746-73ef08879184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:29.553205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:31:29.591567 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e184b2d-ad39-44da-8ccc-5344c54249cd None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:31:29.602784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e184b2d-ad39-44da-8ccc-5344c54249cd None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:31:29.606227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c7d9b-4923-4731-9302-b40b0a51654b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:31:29.606858 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c7d9b-4923-4731-9302-b40b0a51654b service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:29.607137 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c7d9b-4923-4731-9302-b40b0a51654b service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:29.607412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c7d9b-4923-4731-9302-b40b0a51654b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:29.616709 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c7d9b-4923-4731-9302-b40b0a51654b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:31:29.618173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c7d9b-4923-4731-9302-b40b0a51654b service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:31:29.627634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 723/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:31:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:31:32.034533 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15ec80-bf89-4d62-bbbf-afcf0f6d28cd None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:32.035083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15ec80-bf89-4d62-bbbf-afcf0f6d28cd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:32.035083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15ec80-bf89-4d62-bbbf-afcf0f6d28cd None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:32.036492 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:32.042224 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6120c04-f962-4da8-a601-6708392bf169 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:32.042645 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6120c04-f962-4da8-a601-6708392bf169 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:32.042775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6120c04-f962-4da8-a601-6708392bf169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:32.060280 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e6120c04-f962-4da8-a601-6708392bf169 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:32.062045 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e6120c04-f962-4da8-a601-6708392bf169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:32.069774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:31:32.108380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15ec80-bf89-4d62-bbbf-afcf0f6d28cd None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:31:32.119005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15ec80-bf89-4d62-bbbf-afcf0f6d28cd None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:31:32.122483 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4bb88-2a5f-43ec-9ba1-f6802063af13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:31:32.123121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4bb88-2a5f-43ec-9ba1-f6802063af13 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:32.123401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4bb88-2a5f-43ec-9ba1-f6802063af13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:32.123674 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4bb88-2a5f-43ec-9ba1-f6802063af13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:32.133208 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4bb88-2a5f-43ec-9ba1-f6802063af13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:31:32.134641 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4bb88-2a5f-43ec-9ba1-f6802063af13 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:31:32.144715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 722/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:31:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:31:34.748397 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2992a-5a20-436f-b340-f6814bbadc28 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:34.749071 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2992a-5a20-436f-b340-f6814bbadc28 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:34.749071 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2992a-5a20-436f-b340-f6814bbadc28 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:34.750428 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:34.755719 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812224a1-b740-406c-b9ae-a93fddd733f0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:34.756158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812224a1-b740-406c-b9ae-a93fddd733f0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:34.756310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812224a1-b740-406c-b9ae-a93fddd733f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:34.774422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-812224a1-b740-406c-b9ae-a93fddd733f0 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:34.775627 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-812224a1-b740-406c-b9ae-a93fddd733f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:34.783989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:31:34.822861 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2992a-5a20-436f-b340-f6814bbadc28 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:31:34.834195 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2992a-5a20-436f-b340-f6814bbadc28 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:31:34.837672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634923a-a0a0-4db7-89ba-8767e0e885a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:31:34.838297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634923a-a0a0-4db7-89ba-8767e0e885a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:34.838603 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634923a-a0a0-4db7-89ba-8767e0e885a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:34.838871 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634923a-a0a0-4db7-89ba-8767e0e885a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:34.849287 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0634923a-a0a0-4db7-89ba-8767e0e885a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:31:34.850700 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0634923a-a0a0-4db7-89ba-8767e0e885a7 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:31:34.860832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 726/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:31:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:31:37.223741 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9737925-ef84-451f-916b-e851be46aaa1 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:37.224282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9737925-ef84-451f-916b-e851be46aaa1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:37.224282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9737925-ef84-451f-916b-e851be46aaa1 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:37.225662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:37.231358 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bda000-e4c7-4e4c-9c53-1fe239315c0d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:37.231831 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bda000-e4c7-4e4c-9c53-1fe239315c0d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:37.231941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bda000-e4c7-4e4c-9c53-1fe239315c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:37.249054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-79bda000-e4c7-4e4c-9c53-1fe239315c0d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:37.250231 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-79bda000-e4c7-4e4c-9c53-1fe239315c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:37.258178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:31:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:31:37.298047 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9737925-ef84-451f-916b-e851be46aaa1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:31:37.308658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9737925-ef84-451f-916b-e851be46aaa1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:31:37.312640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d7a35-2e26-49a2-a3ff-f3de0e5ad0ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:31:37.313304 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d7a35-2e26-49a2-a3ff-f3de0e5ad0ed service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:37.313579 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d7a35-2e26-49a2-a3ff-f3de0e5ad0ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:37.313847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d7a35-2e26-49a2-a3ff-f3de0e5ad0ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:37.323410 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d7a35-2e26-49a2-a3ff-f3de0e5ad0ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:31:37.324843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d7a35-2e26-49a2-a3ff-f3de0e5ad0ed service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:31:37.334804 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 725/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:31:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:31:39.640169 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccacf20-ed06-45a1-ac3b-c9f36f0143c5 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:39.641172 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccacf20-ed06-45a1-ac3b-c9f36f0143c5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:39.641172 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccacf20-ed06-45a1-ac3b-c9f36f0143c5 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:39.642923 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:31:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:39.649822 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9aea1-7c63-4a0a-9af6-acc1de73f340 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:39.650350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9aea1-7c63-4a0a-9af6-acc1de73f340 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:39.650470 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9aea1-7c63-4a0a-9af6-acc1de73f340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:39.668592 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-22d9aea1-7c63-4a0a-9af6-acc1de73f340 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:39.669851 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-22d9aea1-7c63-4a0a-9af6-acc1de73f340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:39.677837 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:31:39.717045 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccacf20-ed06-45a1-ac3b-c9f36f0143c5 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:31:39.729272 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccacf20-ed06-45a1-ac3b-c9f36f0143c5 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:31:39.732781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2408e-a5fb-42e0-aad6-fb275023e6da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:31:39.734196 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2408e-a5fb-42e0-aad6-fb275023e6da service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:39.734474 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2408e-a5fb-42e0-aad6-fb275023e6da service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:39.734751 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2408e-a5fb-42e0-aad6-fb275023e6da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:39.743818 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d2408e-a5fb-42e0-aad6-fb275023e6da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:31:39.745273 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d2408e-a5fb-42e0-aad6-fb275023e6da service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:31:39.755047 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 729/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:31:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:31:42.102046 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c7e16-8ff0-422c-b6dc-fcca82a34d5f None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:42.102627 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c7e16-8ff0-422c-b6dc-fcca82a34d5f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:42.102627 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c7e16-8ff0-422c-b6dc-fcca82a34d5f None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:42.103964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:42.109956 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60982c-d17f-40e7-bcb5-d3dbcbb3efc6 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:42.110408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60982c-d17f-40e7-bcb5-d3dbcbb3efc6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:42.110521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60982c-d17f-40e7-bcb5-d3dbcbb3efc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:42.127724 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-eb60982c-d17f-40e7-bcb5-d3dbcbb3efc6 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:42.128918 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-eb60982c-d17f-40e7-bcb5-d3dbcbb3efc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:42.136639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:31:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:31:42.175592 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c7e16-8ff0-422c-b6dc-fcca82a34d5f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:31:42.186881 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c7e16-8ff0-422c-b6dc-fcca82a34d5f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:31:42.190485 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da8467-d17d-4173-afd9-55663ef8120c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:31:42.191307 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da8467-d17d-4173-afd9-55663ef8120c service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:42.191586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da8467-d17d-4173-afd9-55663ef8120c service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:42.191860 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da8467-d17d-4173-afd9-55663ef8120c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:42.201294 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da8467-d17d-4173-afd9-55663ef8120c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:31:42.203323 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da8467-d17d-4173-afd9-55663ef8120c service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:31:42.212830 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 728/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:31:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:31:44.507618 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c3b66-978b-4f41-a3f6-d542107e5ca7 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:44.508156 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c3b66-978b-4f41-a3f6-d542107e5ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:44.508156 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c3b66-978b-4f41-a3f6-d542107e5ca7 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:44.509579 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:44.515308 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bebc9d-7085-466a-a5a5-f2a2a182bed0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:44.515742 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bebc9d-7085-466a-a5a5-f2a2a182bed0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:44.515952 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bebc9d-7085-466a-a5a5-f2a2a182bed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:44.522317 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=70 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:31:44.534017 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-38bebc9d-7085-466a-a5a5-f2a2a182bed0 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:44.535201 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-38bebc9d-7085-466a-a5a5-f2a2a182bed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:44.543144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:31:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:31:44.581031 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c3b66-978b-4f41-a3f6-d542107e5ca7 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:31:44.586561 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=73 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:31:44.592049 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c3b66-978b-4f41-a3f6-d542107e5ca7 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:31:44.595479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871c8f0-96e4-40f2-9508-51acc4cbbe7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:31:44.596267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871c8f0-96e4-40f2-9508-51acc4cbbe7c service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:44.596545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871c8f0-96e4-40f2-9508-51acc4cbbe7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:44.596818 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871c8f0-96e4-40f2-9508-51acc4cbbe7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:44.605809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871c8f0-96e4-40f2-9508-51acc4cbbe7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:31:44.607241 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871c8f0-96e4-40f2-9508-51acc4cbbe7c service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:31:44.617567 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 732/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:31:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:31:46.901628 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61ac2a-a0f1-4ed1-9bcf-815c1b3eca3a None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:46.902193 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61ac2a-a0f1-4ed1-9bcf-815c1b3eca3a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:46.902193 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61ac2a-a0f1-4ed1-9bcf-815c1b3eca3a None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:46.903546 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:46.909378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186c3c-1ce9-4444-a9bf-466bdae17f64 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:46.909797 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186c3c-1ce9-4444-a9bf-466bdae17f64 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:46.909914 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186c3c-1ce9-4444-a9bf-466bdae17f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:46.927569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7e186c3c-1ce9-4444-a9bf-466bdae17f64 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:46.933414 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7e186c3c-1ce9-4444-a9bf-466bdae17f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:46.936451 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:31:46.974693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61ac2a-a0f1-4ed1-9bcf-815c1b3eca3a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:31:46.985352 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61ac2a-a0f1-4ed1-9bcf-815c1b3eca3a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:31:46.988930 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dea5bd-0187-4181-8d5c-a008d140ff71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:31:46.989600 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dea5bd-0187-4181-8d5c-a008d140ff71 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:46.989883 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dea5bd-0187-4181-8d5c-a008d140ff71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:46.990160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dea5bd-0187-4181-8d5c-a008d140ff71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:46.999086 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dea5bd-0187-4181-8d5c-a008d140ff71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:31:47.000543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dea5bd-0187-4181-8d5c-a008d140ff71 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:31:47.009831 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 731/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:31:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:31:49.294121 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c4170-036f-4023-921f-40060c646977 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:49.294649 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c4170-036f-4023-921f-40060c646977 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:49.294649 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c4170-036f-4023-921f-40060c646977 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:49.296002 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:49.301387 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d93f59-c813-4ba3-96ad-756ba4f881d0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:49.301865 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d93f59-c813-4ba3-96ad-756ba4f881d0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:49.302108 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d93f59-c813-4ba3-96ad-756ba4f881d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:49.320687 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-83d93f59-c813-4ba3-96ad-756ba4f881d0 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:49.321945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-83d93f59-c813-4ba3-96ad-756ba4f881d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:49.332037 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:31:49.371131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c4170-036f-4023-921f-40060c646977 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:31:49.382837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c4170-036f-4023-921f-40060c646977 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:31:49.386510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ed28b-a93e-40fd-9e4a-48b12180b547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:31:49.387154 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ed28b-a93e-40fd-9e4a-48b12180b547 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:49.387457 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ed28b-a93e-40fd-9e4a-48b12180b547 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:49.387717 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ed28b-a93e-40fd-9e4a-48b12180b547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:49.400117 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ed28b-a93e-40fd-9e4a-48b12180b547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:31:49.402285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ed28b-a93e-40fd-9e4a-48b12180b547 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:31:49.413665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 735/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:31:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:31:51.747638 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92843e-220c-47ab-821a-6030c79d68dc None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:51.748181 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92843e-220c-47ab-821a-6030c79d68dc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:51.748282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92843e-220c-47ab-821a-6030c79d68dc None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:51.749650 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:31:51.755305 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5c5ab-3cbf-427d-8a15-945ec5ab41fa None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:51.755746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5c5ab-3cbf-427d-8a15-945ec5ab41fa None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:51.755899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5c5ab-3cbf-427d-8a15-945ec5ab41fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:51.773522 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e3c5c5ab-3cbf-427d-8a15-945ec5ab41fa None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:31:51.774715 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e3c5c5ab-3cbf-427d-8a15-945ec5ab41fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:31:51.783182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:31:51.820820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92843e-220c-47ab-821a-6030c79d68dc None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:31:51.831574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92843e-220c-47ab-821a-6030c79d68dc None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:31:51.843754 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec7701-c8d7-4ca9-9129-87d0aacca956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:31:51.844398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec7701-c8d7-4ca9-9129-87d0aacca956 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:31:51.844678 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec7701-c8d7-4ca9-9129-87d0aacca956 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:31:51.844950 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec7701-c8d7-4ca9-9129-87d0aacca956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:31:51.854589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec7701-c8d7-4ca9-9129-87d0aacca956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:31:51.856030 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec7701-c8d7-4ca9-9129-87d0aacca956 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:31:51.866045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 734/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:31:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:32:01.768489 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=17 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:32:01.861815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=17 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:32:24.609733 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1004d01-191b-40d0-8b07-a4bb481fac4b None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:24.610371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1004d01-191b-40d0-8b07-a4bb481fac4b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:24.610371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1004d01-191b-40d0-8b07-a4bb481fac4b None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:24.611667 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 737/1471] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:32:24.617475 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74e41b-ccc0-405b-ac13-18455f335859 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:24.617969 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74e41b-ccc0-405b-ac13-18455f335859 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:24.618075 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74e41b-ccc0-405b-ac13-18455f335859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:24.636351 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fc74e41b-ccc0-405b-ac13-18455f335859 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:32:24.637560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fc74e41b-ccc0-405b-ac13-18455f335859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:32:24.646000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 735/1472] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:32:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:32:24.695462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512318ea-9309-44b6-b947-7ed3e6aaae1c None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:24.695765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512318ea-9309-44b6-b947-7ed3e6aaae1c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:24.696055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512318ea-9309-44b6-b947-7ed3e6aaae1c None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:24.697413 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 738/1473] 10.222.0.17 () {54 vars in 962 bytes} [Tue Jun 7 08:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:32:24.703149 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e669dc-89e6-4f93-bd52-125a0aa880ee None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:24.703457 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e669dc-89e6-4f93-bd52-125a0aa880ee None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:24.703743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e669dc-89e6-4f93-bd52-125a0aa880ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:24.723509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-83e669dc-89e6-4f93-bd52-125a0aa880ee None None] MFA Rules not processed for user `c4d9b5e3826049f2800f1b9424977d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:32:24.724820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-83e669dc-89e6-4f93-bd52-125a0aa880ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:32:24.731985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 736/1474] 10.222.0.17 () {58 vars in 1059 bytes} [Tue Jun 7 08:32:24 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:32:24.739973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512318ea-9309-44b6-b947-7ed3e6aaae1c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:32:24.751070 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512318ea-9309-44b6-b947-7ed3e6aaae1c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:32:24.755013 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880a1d5-e919-489f-b72d-99f1c7027019 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4d9b5e3826049f2800f1b9424977d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:32:24.755718 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880a1d5-e919-489f-b72d-99f1c7027019 service glance] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:24.756001 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880a1d5-e919-489f-b72d-99f1c7027019 service glance] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:24.756275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880a1d5-e919-489f-b72d-99f1c7027019 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:24.766089 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880a1d5-e919-489f-b72d-99f1c7027019 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:32:24.767646 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880a1d5-e919-489f-b72d-99f1c7027019 service glance] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:32:24.777848 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 739/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:32:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:32:27.225846 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfab3f-7faa-4395-bd2c-f1633f76300a None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:27.226411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfab3f-7faa-4395-bd2c-f1633f76300a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:27.226411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfab3f-7faa-4395-bd2c-f1633f76300a None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:27.227754 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 737/1476] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:32:27.233274 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81268b7-a6f3-4e0e-a267-cac82a9f63d1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:27.233767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81268b7-a6f3-4e0e-a267-cac82a9f63d1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:27.233875 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81268b7-a6f3-4e0e-a267-cac82a9f63d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:27.253512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a81268b7-a6f3-4e0e-a267-cac82a9f63d1 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:32:27.253830 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a81268b7-a6f3-4e0e-a267-cac82a9f63d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:32:27.262638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 740/1477] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:32:27.287731 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfab3f-7faa-4395-bd2c-f1633f76300a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:32:27.300622 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfab3f-7faa-4395-bd2c-f1633f76300a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:32:27.305171 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed9f77-5516-426c-a830-d963504be3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:32:27.305826 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed9f77-5516-426c-a830-d963504be3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:27.306109 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed9f77-5516-426c-a830-d963504be3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:27.306391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed9f77-5516-426c-a830-d963504be3b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:27.307120 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed9f77-5516-426c-a830-d963504be3b0 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:32:27.308845 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-fbed9f77-5516-426c-a830-d963504be3b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:32:27.308845 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 07 08:32:27.308845 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: Jun 07 08:32:27.310026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed9f77-5516-426c-a830-d963504be3b0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:32:27.330405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 738/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Jun 7 08:32:27 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:32:29.806981 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc8baf-a18c-44fa-8aad-c7f210e262aa None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:29.807561 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc8baf-a18c-44fa-8aad-c7f210e262aa None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:29.807672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc8baf-a18c-44fa-8aad-c7f210e262aa None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:29.809081 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 741/1479] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:32:29.815175 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe598490-3978-4788-913a-95f1e59f702b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:29.815175 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe598490-3978-4788-913a-95f1e59f702b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:29.816365 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe598490-3978-4788-913a-95f1e59f702b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:29.835276 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fe598490-3978-4788-913a-95f1e59f702b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:32:29.836587 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fe598490-3978-4788-913a-95f1e59f702b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:32:29.845110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 739/1480] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:32:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:32:29.891587 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc8baf-a18c-44fa-8aad-c7f210e262aa None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:32:29.906037 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc8baf-a18c-44fa-8aad-c7f210e262aa None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:32:29.911707 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de08d4-3ef4-4cf4-8331-5f4d14843566 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:32:29.912486 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de08d4-3ef4-4cf4-8331-5f4d14843566 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:29.912770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de08d4-3ef4-4cf4-8331-5f4d14843566 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:29.913079 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de08d4-3ef4-4cf4-8331-5f4d14843566 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:29.922817 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de08d4-3ef4-4cf4-8331-5f4d14843566 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:32:29.926626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de08d4-3ef4-4cf4-8331-5f4d14843566 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:32:29.936323 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 742/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:32:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:32:32.693600 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdac5d8-af37-45ec-a773-f19f00c0e550 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:32.694147 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdac5d8-af37-45ec-a773-f19f00c0e550 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:32.694147 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdac5d8-af37-45ec-a773-f19f00c0e550 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:32.695540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 740/1482] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:32:32.701264 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad561c9f-b21d-4b8d-b325-742b403061d4 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:32.701740 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad561c9f-b21d-4b8d-b325-742b403061d4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:32.701836 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad561c9f-b21d-4b8d-b325-742b403061d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:32.719981 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ad561c9f-b21d-4b8d-b325-742b403061d4 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:32:32.721231 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ad561c9f-b21d-4b8d-b325-742b403061d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:32:32.729156 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 743/1483] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:32:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:32:32.767290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdac5d8-af37-45ec-a773-f19f00c0e550 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:32:32.780919 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdac5d8-af37-45ec-a773-f19f00c0e550 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:32:32.785120 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119b5b1-e7fd-49f6-b1db-76406406dbdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:32:32.785757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119b5b1-e7fd-49f6-b1db-76406406dbdf service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:32.786035 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119b5b1-e7fd-49f6-b1db-76406406dbdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:32.786309 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119b5b1-e7fd-49f6-b1db-76406406dbdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:32.796072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119b5b1-e7fd-49f6-b1db-76406406dbdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:32:32.797538 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119b5b1-e7fd-49f6-b1db-76406406dbdf service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:32:32.806888 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 741/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:32:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:32:35.122375 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a561f-6e3f-427f-8438-41ff7d6fa970 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:35.122941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a561f-6e3f-427f-8438-41ff7d6fa970 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:35.122941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a561f-6e3f-427f-8438-41ff7d6fa970 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:35.124224 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 744/1485] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:32:35.129749 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a344a-ebe7-4c86-ba01-c67fca970998 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:35.130265 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a344a-ebe7-4c86-ba01-c67fca970998 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:35.130361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a344a-ebe7-4c86-ba01-c67fca970998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:35.156159 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-458a344a-ebe7-4c86-ba01-c67fca970998 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:32:35.159866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-458a344a-ebe7-4c86-ba01-c67fca970998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:32:35.166855 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 742/1486] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:32:35.206060 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a561f-6e3f-427f-8438-41ff7d6fa970 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:32:35.217963 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a561f-6e3f-427f-8438-41ff7d6fa970 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:32:35.221795 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ecda64-f6d4-468b-ad09-f8be434fbc77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:32:35.222716 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecda64-f6d4-468b-ad09-f8be434fbc77 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:35.223193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecda64-f6d4-468b-ad09-f8be434fbc77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:35.223556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecda64-f6d4-468b-ad09-f8be434fbc77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:35.233784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ecda64-f6d4-468b-ad09-f8be434fbc77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:32:35.235248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ecda64-f6d4-468b-ad09-f8be434fbc77 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:32:35.247128 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 745/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:32:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:32:37.734272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83d872-9347-4b55-a8f5-2c1f41df51a8 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:37.734946 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83d872-9347-4b55-a8f5-2c1f41df51a8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:37.734946 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83d872-9347-4b55-a8f5-2c1f41df51a8 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:37.736229 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 743/1488] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:32:37.741763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4395ba-616f-4ce9-9ac0-56dbb0cfcbdb None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:37.742256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4395ba-616f-4ce9-9ac0-56dbb0cfcbdb None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:37.742361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4395ba-616f-4ce9-9ac0-56dbb0cfcbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:37.760677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7a4395ba-616f-4ce9-9ac0-56dbb0cfcbdb None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:32:37.761888 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7a4395ba-616f-4ce9-9ac0-56dbb0cfcbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:32:37.769942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 746/1489] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:32:37.816708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83d872-9347-4b55-a8f5-2c1f41df51a8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:32:37.828346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83d872-9347-4b55-a8f5-2c1f41df51a8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:32:37.832032 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d99a9-7179-4e31-8f5a-5fbdd75dd7c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:32:37.832704 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d99a9-7179-4e31-8f5a-5fbdd75dd7c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:37.833032 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d99a9-7179-4e31-8f5a-5fbdd75dd7c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:37.833736 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d99a9-7179-4e31-8f5a-5fbdd75dd7c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:37.842721 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2d99a9-7179-4e31-8f5a-5fbdd75dd7c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:32:37.844161 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2d99a9-7179-4e31-8f5a-5fbdd75dd7c2 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:32:37.854102 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 744/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:32:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:32:40.730324 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff6b7b-ea3a-4a58-ab84-e034f25ab418 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:40.730942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff6b7b-ea3a-4a58-ab84-e034f25ab418 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:40.730942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff6b7b-ea3a-4a58-ab84-e034f25ab418 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:40.732253 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 747/1491] 10.222.0.17 () {54 vars in 926 bytes} [Tue Jun 7 08:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:32:40.737889 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fe77b-644d-4e99-b373-86692bb72766 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:40.738549 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fe77b-644d-4e99-b373-86692bb72766 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:40.738549 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fe77b-644d-4e99-b373-86692bb72766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:40.756159 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-767fe77b-644d-4e99-b373-86692bb72766 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:32:40.757423 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-767fe77b-644d-4e99-b373-86692bb72766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:32:40.765840 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 745/1492] 10.222.0.17 () {58 vars in 1023 bytes} [Tue Jun 7 08:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:32:40.805306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff6b7b-ea3a-4a58-ab84-e034f25ab418 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:32:40.815949 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff6b7b-ea3a-4a58-ab84-e034f25ab418 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:32:40.819510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b6561-5a8d-4329-93d1-84063399f936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:32:40.820141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b6561-5a8d-4329-93d1-84063399f936 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:32:40.820417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b6561-5a8d-4329-93d1-84063399f936 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:32:40.820684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b6561-5a8d-4329-93d1-84063399f936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:32:40.834706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295b6561-5a8d-4329-93d1-84063399f936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:32:40.836144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295b6561-5a8d-4329-93d1-84063399f936 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:32:40.846535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 748/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:32:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:32:50.751023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=33 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:32:50.841432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=33 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:34:18.406602 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd4db8-b1c6-4df8-92d5-e22bb252f944 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:18.407245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd4db8-b1c6-4df8-92d5-e22bb252f944 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:18.407245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd4db8-b1c6-4df8-92d5-e22bb252f944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:18.425976 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a0fd4db8-b1c6-4df8-92d5-e22bb252f944 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:34:18.427164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a0fd4db8-b1c6-4df8-92d5-e22bb252f944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:18.435585 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 746/1494] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:34:18.444488 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d2d03-8b41-4570-8080-aa0d1b282762 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:18.445093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d2d03-8b41-4570-8080-aa0d1b282762 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:18.445093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d2d03-8b41-4570-8080-aa0d1b282762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:18.463621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-5e9d2d03-8b41-4570-8080-aa0d1b282762 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:34:18.465006 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5e9d2d03-8b41-4570-8080-aa0d1b282762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:18.472979 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 749/1495] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:34:18.477728 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd4db8-b1c6-4df8-92d5-e22bb252f944 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:34:18.488170 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd4db8-b1c6-4df8-92d5-e22bb252f944 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:34:18.492973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e53fac-f26f-46e7-bcf4-d00d6131f678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:34:18.493666 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e53fac-f26f-46e7-bcf4-d00d6131f678 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:18.493941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e53fac-f26f-46e7-bcf4-d00d6131f678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:18.494214 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e53fac-f26f-46e7-bcf4-d00d6131f678 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:18.494930 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e53fac-f26f-46e7-bcf4-d00d6131f678 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:34:18.495229 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e53fac-f26f-46e7-bcf4-d00d6131f678 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:34:18.496236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-54e53fac-f26f-46e7-bcf4-d00d6131f678 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:34:18.496236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 07 08:34:18.496236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: Jun 07 08:34:18.497143 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e53fac-f26f-46e7-bcf4-d00d6131f678 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:34:18.511722 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 747/1496] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:34:18.520886 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d2d03-8b41-4570-8080-aa0d1b282762 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:34:18.530192 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d2d03-8b41-4570-8080-aa0d1b282762 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:34:18.534807 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a261980-2d1b-401d-b9d1-ab64308597f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:34:18.535495 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a261980-2d1b-401d-b9d1-ab64308597f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:18.535774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a261980-2d1b-401d-b9d1-ab64308597f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:18.536076 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a261980-2d1b-401d-b9d1-ab64308597f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:18.536779 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a261980-2d1b-401d-b9d1-ab64308597f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:34:18.538872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: WARNING py.warnings [None req-8a261980-2d1b-401d-b9d1-ab64308597f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:34:18.538872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: warnings.warn(msg) Jun 07 08:34:18.538872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: Jun 07 08:34:18.540184 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a261980-2d1b-401d-b9d1-ab64308597f1 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:34:18.638379 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 750/1497] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:34:18 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:34:18.643206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e53fac-f26f-46e7-bcf4-d00d6131f678 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:34:18.652935 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e53fac-f26f-46e7-bcf4-d00d6131f678 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:34:18.709618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bb547-3f22-47ef-8ebc-322a2294a593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:34:18.710192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bb547-3f22-47ef-8ebc-322a2294a593 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:18.710751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bb547-3f22-47ef-8ebc-322a2294a593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:18.711027 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bb547-3f22-47ef-8ebc-322a2294a593 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:18.711700 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74bb547-3f22-47ef-8ebc-322a2294a593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:34:18.712732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-a74bb547-3f22-47ef-8ebc-322a2294a593 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:34:18.712732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 07 08:34:18.712732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: Jun 07 08:34:18.713961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74bb547-3f22-47ef-8ebc-322a2294a593 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:34:18.798417 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 748/1498] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:34:18 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:34:18.802588 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a261980-2d1b-401d-b9d1-ab64308597f1 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:34:18.814494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a261980-2d1b-401d-b9d1-ab64308597f1 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:34:18.819121 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6efef3-0ca2-4dd3-b269-f91a30fbaaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:34:18.819785 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6efef3-0ca2-4dd3-b269-f91a30fbaaee admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:18.820063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6efef3-0ca2-4dd3-b269-f91a30fbaaee admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:18.820333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6efef3-0ca2-4dd3-b269-f91a30fbaaee admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:18.821093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6efef3-0ca2-4dd3-b269-f91a30fbaaee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:34:18.822114 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: WARNING py.warnings [None req-8c6efef3-0ca2-4dd3-b269-f91a30fbaaee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:34:18.822114 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: warnings.warn(msg) Jun 07 08:34:18.822114 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: Jun 07 08:34:18.823002 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6efef3-0ca2-4dd3-b269-f91a30fbaaee admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:34:18.835813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 751/1499] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:34:18.839874 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bb547-3f22-47ef-8ebc-322a2294a593 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:34:18.850457 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bb547-3f22-47ef-8ebc-322a2294a593 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:34:18.857464 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f629f-27cf-433a-ac74-30b87996147e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:34:18.858129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f629f-27cf-433a-ac74-30b87996147e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:18.858413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f629f-27cf-433a-ac74-30b87996147e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:18.858691 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f629f-27cf-433a-ac74-30b87996147e admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:18.859924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07f629f-27cf-433a-ac74-30b87996147e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:34:18.861153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-d07f629f-27cf-433a-ac74-30b87996147e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:34:18.861153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 07 08:34:18.861153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: Jun 07 08:34:18.862060 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07f629f-27cf-433a-ac74-30b87996147e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:34:18.883985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 749/1500] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:34:18.888208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6efef3-0ca2-4dd3-b269-f91a30fbaaee admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:34:18.898360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6efef3-0ca2-4dd3-b269-f91a30fbaaee admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:34:18.902936 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e0042-2400-4158-a79b-d003162bc1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:34:18.903807 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e0042-2400-4158-a79b-d003162bc1c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:18.904086 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e0042-2400-4158-a79b-d003162bc1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:18.904482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e0042-2400-4158-a79b-d003162bc1c3 admin admin] PATH_INFO: `/v3/projects/91b5f3fa41264096b4218236978bd553/users/53e0634d75fb402fb7415a2f8f91700c/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:18.919712 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e0042-2400-4158-a79b-d003162bc1c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91b5f3fa41264096b4218236978bd553, user_id=53e0634d75fb402fb7415a2f8f91700c, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:34:18.920801 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: WARNING py.warnings [None req-994e0042-2400-4158-a79b-d003162bc1c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:34:18.920801 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: warnings.warn(msg) Jun 07 08:34:18.920801 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: Jun 07 08:34:18.923314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e0042-2400-4158-a79b-d003162bc1c3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:34:19.016344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 752/1501] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:34:18 2022] PUT /identity/v3/projects/91b5f3fa41264096b4218236978bd553/users/53e0634d75fb402fb7415a2f8f91700c/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:34:19.027901 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d88789-c5fd-4539-8b2b-1157464dc6ca None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:19.028230 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d88789-c5fd-4539-8b2b-1157464dc6ca None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:19.028507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d88789-c5fd-4539-8b2b-1157464dc6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:19.055511 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-62d88789-c5fd-4539-8b2b-1157464dc6ca None None] MFA Rules not processed for user `53e0634d75fb402fb7415a2f8f91700c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:34:19.056711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-62d88789-c5fd-4539-8b2b-1157464dc6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:19.120876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 750/1502] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:34:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:34:19.135218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b01bd-d47b-4e05-aded-e4de60c268d5 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:19.135505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b01bd-d47b-4e05-aded-e4de60c268d5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:19.135779 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b01bd-d47b-4e05-aded-e4de60c268d5 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:19.137182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 753/1503] 10.222.0.17 () {54 vars in 963 bytes} [Tue Jun 7 08:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:34:19.142738 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fecb95-f8d8-48f4-81d8-93a15e163208 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:19.143034 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fecb95-f8d8-48f4-81d8-93a15e163208 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:19.143313 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fecb95-f8d8-48f4-81d8-93a15e163208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:19.162487 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-65fecb95-f8d8-48f4-81d8-93a15e163208 None None] MFA Rules not processed for user `85ea2962e7094f8c878d95680764c4e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:34:19.163966 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-65fecb95-f8d8-48f4-81d8-93a15e163208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:19.210739 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:34:19.218124 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 751/1504] 10.222.0.17 () {58 vars in 1060 bytes} [Tue Jun 7 08:34:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:34:19.225939 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b01bd-d47b-4e05-aded-e4de60c268d5 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:34:19.240720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b01bd-d47b-4e05-aded-e4de60c268d5 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:34:19.244323 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d84b2-7308-4a1d-80af-5e86c005509d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:34:19.244964 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d84b2-7308-4a1d-80af-5e86c005509d service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:19.245276 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d84b2-7308-4a1d-80af-5e86c005509d service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:19.245549 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d84b2-7308-4a1d-80af-5e86c005509d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:19.254999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d84b2-7308-4a1d-80af-5e86c005509d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:34:19.258765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d84b2-7308-4a1d-80af-5e86c005509d service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:34:19.266815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 754/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:34:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:34:19.308573 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75512849-3403-49b0-ac95-8f6b9eff67eb None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:19.308889 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75512849-3403-49b0-ac95-8f6b9eff67eb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:19.309216 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75512849-3403-49b0-ac95-8f6b9eff67eb None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:19.310494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 752/1506] 10.222.0.17 () {52 vars in 829 bytes} [Tue Jun 7 08:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:34:19.314585 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d84b2-7308-4a1d-80af-5e86c005509d service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:34:19.376892 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d84b2-7308-4a1d-80af-5e86c005509d service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:34:19.381363 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c414e6d-ebac-4631-92a6-9a488dd3d878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:34:19.382237 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c414e6d-ebac-4631-92a6-9a488dd3d878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:19.382515 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c414e6d-ebac-4631-92a6-9a488dd3d878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:19.382786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c414e6d-ebac-4631-92a6-9a488dd3d878 admin admin] PATH_INFO: `/v3/users/53e0634d75fb402fb7415a2f8f91700c` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:19.383966 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c414e6d-ebac-4631-92a6-9a488dd3d878 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53e0634d75fb402fb7415a2f8f91700c)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:34:19.384973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: WARNING py.warnings [None req-0c414e6d-ebac-4631-92a6-9a488dd3d878 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:34:19.384973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: warnings.warn(msg) Jun 07 08:34:19.384973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: Jun 07 08:34:19.386202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c414e6d-ebac-4631-92a6-9a488dd3d878 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:34:19.639576 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0c414e6d-ebac-4631-92a6-9a488dd3d878 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53e0634d75fb402fb7415a2f8f91700c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:34:19.640249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0c414e6d-ebac-4631-92a6-9a488dd3d878 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53e0634d75fb402fb7415a2f8f91700c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:34:19.652652 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0c414e6d-ebac-4631-92a6-9a488dd3d878 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53e0634d75fb402fb7415a2f8f91700c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:34:19.653317 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0c414e6d-ebac-4631-92a6-9a488dd3d878 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53e0634d75fb402fb7415a2f8f91700c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:34:19.711060 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0c414e6d-ebac-4631-92a6-9a488dd3d878 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53e0634d75fb402fb7415a2f8f91700c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:34:19.729061 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:34:19.737516 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 755/1507] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:34:19 2022] DELETE /identity/v3/users/53e0634d75fb402fb7415a2f8f91700c => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:34:19.742066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7f949-304c-4592-89a3-214667b7c090 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:19.742557 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7f949-304c-4592-89a3-214667b7c090 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:19.742663 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7f949-304c-4592-89a3-214667b7c090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:19.760798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-09f7f949-304c-4592-89a3-214667b7c090 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:34:19.762041 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-09f7f949-304c-4592-89a3-214667b7c090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:19.824105 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 753/1508] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:34:19.830877 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c414e6d-ebac-4631-92a6-9a488dd3d878 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:34:19.841591 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0c414e6d-ebac-4631-92a6-9a488dd3d878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:19.890749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c414e6d-ebac-4631-92a6-9a488dd3d878 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:34:19.894183 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f51a97-fa35-4016-aed8-5de2d2ec5958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:34:19.894830 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f51a97-fa35-4016-aed8-5de2d2ec5958 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:19.895131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f51a97-fa35-4016-aed8-5de2d2ec5958 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:19.895407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f51a97-fa35-4016-aed8-5de2d2ec5958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:19.904473 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f51a97-fa35-4016-aed8-5de2d2ec5958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:34:19.905987 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f51a97-fa35-4016-aed8-5de2d2ec5958 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:34:19.916513 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 756/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:34:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:34:20.490759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7f949-304c-4592-89a3-214667b7c090 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:34:20.492191 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-09f7f949-304c-4592-89a3-214667b7c090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:34:20.505904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7f949-304c-4592-89a3-214667b7c090 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:34:20.511208 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1923a9-59dd-4950-91be-a66c3b5a0013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:34:20.512027 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1923a9-59dd-4950-91be-a66c3b5a0013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:34:20.512306 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1923a9-59dd-4950-91be-a66c3b5a0013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:34:20.512580 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1923a9-59dd-4950-91be-a66c3b5a0013 admin admin] PATH_INFO: `/v3/projects/91b5f3fa41264096b4218236978bd553` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:34:20.515027 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1923a9-59dd-4950-91be-a66c3b5a0013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91b5f3fa41264096b4218236978bd553)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:34:20.516849 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-2b1923a9-59dd-4950-91be-a66c3b5a0013 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:34:20.516849 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 07 08:34:20.516849 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: Jun 07 08:34:20.518083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1923a9-59dd-4950-91be-a66c3b5a0013 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:34:20.637226 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b1923a9-59dd-4950-91be-a66c3b5a0013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91b5f3fa41264096b4218236978bd553', 'request_id': 'req-2b1923a9-59dd-4950-91be-a66c3b5a0013'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:34:20.643764 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b1923a9-59dd-4950-91be-a66c3b5a0013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91b5f3fa41264096b4218236978bd553', 'request_id': 'req-2b1923a9-59dd-4950-91be-a66c3b5a0013'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:34:20.658369 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b1923a9-59dd-4950-91be-a66c3b5a0013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91b5f3fa41264096b4218236978bd553', 'request_id': 'req-2b1923a9-59dd-4950-91be-a66c3b5a0013'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:34:20.762648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b1923a9-59dd-4950-91be-a66c3b5a0013 admin admin] The token cache is being invalidate because project 91b5f3fa41264096b4218236978bd553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:34:20.763314 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b1923a9-59dd-4950-91be-a66c3b5a0013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:34:20.763755 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b1923a9-59dd-4950-91be-a66c3b5a0013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:34:20.765401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 754/1510] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:34:20 2022] DELETE /identity/v3/projects/91b5f3fa41264096b4218236978bd553 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:34:29.911745 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:34:30.759082 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:01.177439 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a61c5-736c-49ed-994f-c3c1919e50b3 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:01.178017 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a61c5-736c-49ed-994f-c3c1919e50b3 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:01.178017 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a61c5-736c-49ed-994f-c3c1919e50b3 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:01.179368 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:36:01.184310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812ee2f-148e-4046-9602-a6f2f64cf6cb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:01.184981 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812ee2f-148e-4046-9602-a6f2f64cf6cb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:01.184981 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812ee2f-148e-4046-9602-a6f2f64cf6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:01.203847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3812ee2f-148e-4046-9602-a6f2f64cf6cb None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:01.205079 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3812ee2f-148e-4046-9602-a6f2f64cf6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:01.262759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:01.436417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a61c5-736c-49ed-994f-c3c1919e50b3 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:01.438332 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-155a61c5-736c-49ed-994f-c3c1919e50b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:01.527926 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a61c5-736c-49ed-994f-c3c1919e50b3 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:01.533707 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c19ec1-3466-4949-8187-789420e06d7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:01.534384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c19ec1-3466-4949-8187-789420e06d7e service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:01.534659 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c19ec1-3466-4949-8187-789420e06d7e service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:01.534932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c19ec1-3466-4949-8187-789420e06d7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:01.543979 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c19ec1-3466-4949-8187-789420e06d7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:01.545679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c19ec1-3466-4949-8187-789420e06d7e service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:01.555925 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 758/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:02.298583 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d437d32-87f7-4b80-b390-edd19722e290 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:02.299144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d437d32-87f7-4b80-b390-edd19722e290 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:02.299144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d437d32-87f7-4b80-b390-edd19722e290 None None] PATH_INFO: `/v3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:02.300368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 756/1514] 10.222.0.18 () {54 vars in 925 bytes} [Tue Jun 7 08:36:02 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:36:02.307044 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06672a-99e0-40d0-951b-e6e92d71a8b1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:02.307576 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06672a-99e0-40d0-951b-e6e92d71a8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:02.307691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06672a-99e0-40d0-951b-e6e92d71a8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:02.327010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-cc06672a-99e0-40d0-951b-e6e92d71a8b1 None None] MFA Rules not processed for user `1fc0d7bd7e4547f4a8d90ffc2c2e1d03`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:02.328299 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cc06672a-99e0-40d0-951b-e6e92d71a8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:02.392812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 759/1515] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Jun 7 08:36:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:02.446682 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e02e6b-b678-4fbb-bbf2-c572be251b24 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:02.446900 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e02e6b-b678-4fbb-bbf2-c572be251b24 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:02.447176 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e02e6b-b678-4fbb-bbf2-c572be251b24 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:02.448603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 757/1516] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:36:02.453372 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d83b8-5762-4c6f-91d9-7e811196de9d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:02.453665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d83b8-5762-4c6f-91d9-7e811196de9d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:02.453971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d83b8-5762-4c6f-91d9-7e811196de9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:02.472772 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-149d83b8-5762-4c6f-91d9-7e811196de9d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:02.474031 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-149d83b8-5762-4c6f-91d9-7e811196de9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:02.482833 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 760/1517] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:02.636882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e02e6b-b678-4fbb-bbf2-c572be251b24 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:02.653717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e02e6b-b678-4fbb-bbf2-c572be251b24 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:02.659256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23384e13-8b33-415f-9f0b-1a953d3ce33a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:02.659903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23384e13-8b33-415f-9f0b-1a953d3ce33a service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:02.660189 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23384e13-8b33-415f-9f0b-1a953d3ce33a service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:02.660468 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23384e13-8b33-415f-9f0b-1a953d3ce33a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:02.669813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23384e13-8b33-415f-9f0b-1a953d3ce33a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:02.671609 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23384e13-8b33-415f-9f0b-1a953d3ce33a service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:02.681852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 758/1518] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:06.555674 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3a502-c8f2-4f86-9777-59f5bc20cd25 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:06.556516 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3a502-c8f2-4f86-9777-59f5bc20cd25 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:06.556516 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3a502-c8f2-4f86-9777-59f5bc20cd25 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:06.557650 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 761/1519] 10.222.0.17 () {54 vars in 960 bytes} [Tue Jun 7 08:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:36:06.563115 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be0252-013c-43bf-8c73-16545c152c78 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:06.563605 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be0252-013c-43bf-8c73-16545c152c78 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:06.563721 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be0252-013c-43bf-8c73-16545c152c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:06.581921 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-45be0252-013c-43bf-8c73-16545c152c78 None None] MFA Rules not processed for user `1fc0d7bd7e4547f4a8d90ffc2c2e1d03`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:06.583140 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-45be0252-013c-43bf-8c73-16545c152c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:06.592269 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 759/1520] 10.222.0.17 () {58 vars in 1057 bytes} [Tue Jun 7 08:36:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:06.600860 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3a502-c8f2-4f86-9777-59f5bc20cd25 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:06.615027 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3a502-c8f2-4f86-9777-59f5bc20cd25 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:06.620644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bebef-acec-4723-b2aa-d9b324bed185 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc0d7bd7e4547f4a8d90ffc2c2e1d03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:06.621328 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bebef-acec-4723-b2aa-d9b324bed185 service placement] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:06.621609 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bebef-acec-4723-b2aa-d9b324bed185 service placement] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:06.621884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bebef-acec-4723-b2aa-d9b324bed185 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:06.630719 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0bebef-acec-4723-b2aa-d9b324bed185 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:06.632901 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0bebef-acec-4723-b2aa-d9b324bed185 service placement] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:06.642864 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 762/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:13.965953 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01560ace-c192-4d8e-9f2d-9246fbcabe23 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:13.966698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01560ace-c192-4d8e-9f2d-9246fbcabe23 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:13.966698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01560ace-c192-4d8e-9f2d-9246fbcabe23 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:13.968046 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:36:13.978842 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0764b72-7aa9-4325-9c1d-3720c753d2e4 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:13.979402 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0764b72-7aa9-4325-9c1d-3720c753d2e4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:13.979506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0764b72-7aa9-4325-9c1d-3720c753d2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:13.998451 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b0764b72-7aa9-4325-9c1d-3720c753d2e4 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:13.999645 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b0764b72-7aa9-4325-9c1d-3720c753d2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:14.007607 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:36:14.164374 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01560ace-c192-4d8e-9f2d-9246fbcabe23 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:14.178926 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01560ace-c192-4d8e-9f2d-9246fbcabe23 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:14.185056 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d0612-672e-4be0-9795-e620d1086f90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:14.186213 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d0612-672e-4be0-9795-e620d1086f90 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:14.186672 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d0612-672e-4be0-9795-e620d1086f90 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:14.187181 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d0612-672e-4be0-9795-e620d1086f90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:14.196309 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d0612-672e-4be0-9795-e620d1086f90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:14.197977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d0612-672e-4be0-9795-e620d1086f90 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:14.208059 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 761/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:15.088555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15684411-856f-425e-8a2e-b2b0bce10b4f None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:15.089155 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15684411-856f-425e-8a2e-b2b0bce10b4f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:15.089155 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15684411-856f-425e-8a2e-b2b0bce10b4f None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:15.090616 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 08:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:36:15.098885 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7d8d6-142e-4168-8f8c-e9b94973b06f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:15.099508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7d8d6-142e-4168-8f8c-e9b94973b06f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:15.099508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7d8d6-142e-4168-8f8c-e9b94973b06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:15.119979 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-17f7d8d6-142e-4168-8f8c-e9b94973b06f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:15.121222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-17f7d8d6-142e-4168-8f8c-e9b94973b06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:15.129544 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 08:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:36:15.283591 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15684411-856f-425e-8a2e-b2b0bce10b4f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:15.298882 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15684411-856f-425e-8a2e-b2b0bce10b4f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:15.304972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b4c23a-48ac-40af-8044-af9a5754676f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:15.305655 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4c23a-48ac-40af-8044-af9a5754676f service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:15.306078 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4c23a-48ac-40af-8044-af9a5754676f service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:15.306363 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4c23a-48ac-40af-8044-af9a5754676f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:15.315118 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b4c23a-48ac-40af-8044-af9a5754676f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:15.316750 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b4c23a-48ac-40af-8044-af9a5754676f service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:15.326723 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 765/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:19.682510 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83efb4-9e72-4b64-b980-b240d4730ee6 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:19.683140 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83efb4-9e72-4b64-b980-b240d4730ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:19.683140 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83efb4-9e72-4b64-b980-b240d4730ee6 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:19.684621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:36:19.690697 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270f92e-48d3-4bae-8991-fdbd676f80be None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:19.691227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270f92e-48d3-4bae-8991-fdbd676f80be None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:19.691334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270f92e-48d3-4bae-8991-fdbd676f80be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:19.710974 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7270f92e-48d3-4bae-8991-fdbd676f80be None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:19.712242 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7270f92e-48d3-4bae-8991-fdbd676f80be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:19.720340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:19.759295 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83efb4-9e72-4b64-b980-b240d4730ee6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:19.761090 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9d83efb4-9e72-4b64-b980-b240d4730ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:19.820779 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83efb4-9e72-4b64-b980-b240d4730ee6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:19.824295 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa078a-b9d7-4824-90b1-294d0a59174c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:19.824937 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa078a-b9d7-4824-90b1-294d0a59174c service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:19.825249 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa078a-b9d7-4824-90b1-294d0a59174c service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:19.825524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa078a-b9d7-4824-90b1-294d0a59174c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:19.835018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa078a-b9d7-4824-90b1-294d0a59174c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:19.836449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa078a-b9d7-4824-90b1-294d0a59174c service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:19.845888 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 764/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:22.135731 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b8bc8-08a3-47af-bdf2-fc9b499c4dd8 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:22.136351 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b8bc8-08a3-47af-bdf2-fc9b499c4dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:22.136351 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b8bc8-08a3-47af-bdf2-fc9b499c4dd8 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:22.137691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:36:22.143398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2955d-ee9a-4ca4-a0d9-ba57e6ee0075 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:22.143876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2955d-ee9a-4ca4-a0d9-ba57e6ee0075 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:22.144002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2955d-ee9a-4ca4-a0d9-ba57e6ee0075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:22.163984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e4c2955d-ee9a-4ca4-a0d9-ba57e6ee0075 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:22.165293 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e4c2955d-ee9a-4ca4-a0d9-ba57e6ee0075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:22.173325 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:22.210254 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b8bc8-08a3-47af-bdf2-fc9b499c4dd8 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:22.221088 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b8bc8-08a3-47af-bdf2-fc9b499c4dd8 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:22.225101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fb10a-8559-4d3f-a192-fa22c8851cd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:22.225744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fb10a-8559-4d3f-a192-fa22c8851cd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:22.226022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fb10a-8559-4d3f-a192-fa22c8851cd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:22.226294 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fb10a-8559-4d3f-a192-fa22c8851cd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:22.235709 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31fb10a-8559-4d3f-a192-fa22c8851cd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:22.237181 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31fb10a-8559-4d3f-a192-fa22c8851cd8 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:22.248428 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 768/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:26.108265 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08741a-31c1-4884-94a1-4ab4a0b9e1ba None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:26.108876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08741a-31c1-4884-94a1-4ab4a0b9e1ba None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:26.108876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08741a-31c1-4884-94a1-4ab4a0b9e1ba None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:26.110261 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:36:26.115939 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c230103-11a5-4a6f-808f-ed48328e483e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:26.116391 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c230103-11a5-4a6f-808f-ed48328e483e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:26.116495 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c230103-11a5-4a6f-808f-ed48328e483e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:26.144382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9c230103-11a5-4a6f-808f-ed48328e483e None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:26.145638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9c230103-11a5-4a6f-808f-ed48328e483e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:26.153601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:26.427015 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08741a-31c1-4884-94a1-4ab4a0b9e1ba None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:26.428862 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-db08741a-31c1-4884-94a1-4ab4a0b9e1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:26.490051 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08741a-31c1-4884-94a1-4ab4a0b9e1ba None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:26.494783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9126e91-3049-4997-b8d8-586d9b3d184e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:26.495443 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9126e91-3049-4997-b8d8-586d9b3d184e service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:26.495753 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9126e91-3049-4997-b8d8-586d9b3d184e service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:26.496032 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9126e91-3049-4997-b8d8-586d9b3d184e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:26.510681 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9126e91-3049-4997-b8d8-586d9b3d184e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:26.512125 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9126e91-3049-4997-b8d8-586d9b3d184e service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:26.519904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:26.525143 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 767/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:28.824954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db960e-8ad4-4a7b-934e-ccfa16e711f3 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:28.825563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db960e-8ad4-4a7b-934e-ccfa16e711f3 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:28.825563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db960e-8ad4-4a7b-934e-ccfa16e711f3 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:28.826899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 08:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:36:28.832202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc442c1f-05ac-450b-8ea1-4196378b12d9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:28.832665 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc442c1f-05ac-450b-8ea1-4196378b12d9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:28.832770 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc442c1f-05ac-450b-8ea1-4196378b12d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:28.851823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fc442c1f-05ac-450b-8ea1-4196378b12d9 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:36:28.853581 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fc442c1f-05ac-450b-8ea1-4196378b12d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:28.861861 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 08:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:36:29.140573 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db960e-8ad4-4a7b-934e-ccfa16e711f3 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:36:29.142341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-94db960e-8ad4-4a7b-934e-ccfa16e711f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:36:29.155796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db960e-8ad4-4a7b-934e-ccfa16e711f3 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:36:29.159345 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcd4ce-d194-492b-80e9-f69c08f0efa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:36:29.159985 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcd4ce-d194-492b-80e9-f69c08f0efa5 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:36:29.160265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcd4ce-d194-492b-80e9-f69c08f0efa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:36:29.160541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcd4ce-d194-492b-80e9-f69c08f0efa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:36:29.170435 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcd4ce-d194-492b-80e9-f69c08f0efa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:36:29.171877 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcd4ce-d194-492b-80e9-f69c08f0efa5 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:36:29.182436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 771/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:36:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:36:38.846815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:36:39.177392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:03.933195 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81efb94-7278-4993-bed7-042dbbb51e4a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:03.933857 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81efb94-7278-4993-bed7-042dbbb51e4a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:03.933857 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81efb94-7278-4993-bed7-042dbbb51e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:03.981470 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c81efb94-7278-4993-bed7-042dbbb51e4a None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:03.982721 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c81efb94-7278-4993-bed7-042dbbb51e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:03.993545 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 769/1540] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:04.002555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1415132-bdae-4eae-a6eb-013aa86249f3 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.003149 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1415132-bdae-4eae-a6eb-013aa86249f3 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.003149 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1415132-bdae-4eae-a6eb-013aa86249f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.029665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d1415132-bdae-4eae-a6eb-013aa86249f3 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:04.031149 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977ba12-b791-4300-8e1d-5ab8e7903130 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.031440 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977ba12-b791-4300-8e1d-5ab8e7903130 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.031846 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977ba12-b791-4300-8e1d-5ab8e7903130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.041869 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d1415132-bdae-4eae-a6eb-013aa86249f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:04.067570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 772/1541] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:04.073497 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1415132-bdae-4eae-a6eb-013aa86249f3 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:04.078408 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f977ba12-b791-4300-8e1d-5ab8e7903130 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:04.087728 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f977ba12-b791-4300-8e1d-5ab8e7903130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:04.098054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1415132-bdae-4eae-a6eb-013aa86249f3 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:04.103570 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 770/1542] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:38:04.114727 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d6785-89f8-4270-a34f-aeb1040eadb3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.114948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e3314-159e-4ab1-8f1b-92ebd8831015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:04.115435 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d6785-89f8-4270-a34f-aeb1040eadb3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.115574 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e3314-159e-4ab1-8f1b-92ebd8831015 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.115718 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e3314-159e-4ab1-8f1b-92ebd8831015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.116001 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e3314-159e-4ab1-8f1b-92ebd8831015 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.116332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d6785-89f8-4270-a34f-aeb1040eadb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.119208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e3314-159e-4ab1-8f1b-92ebd8831015 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:04.119208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e3314-159e-4ab1-8f1b-92ebd8831015 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:04.119208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: WARNING py.warnings [None req-e05e3314-159e-4ab1-8f1b-92ebd8831015 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:38:04.119208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: warnings.warn(msg) Jun 07 08:38:04.119208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: Jun 07 08:38:04.119749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e3314-159e-4ab1-8f1b-92ebd8831015 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:04.164111 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 773/1543] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:04.166251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-382d6785-89f8-4270-a34f-aeb1040eadb3 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:04.167573 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-382d6785-89f8-4270-a34f-aeb1040eadb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:04.172064 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e3314-159e-4ab1-8f1b-92ebd8831015 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:04.194629 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 771/1544] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 07 08:38:04.205794 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d6785-89f8-4270-a34f-aeb1040eadb3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:04.209611 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e3314-159e-4ab1-8f1b-92ebd8831015 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:04.225404 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652a368-be88-4571-88a8-c0a55b8a6a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:04.226872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652a368-be88-4571-88a8-c0a55b8a6a81 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.227285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d6785-89f8-4270-a34f-aeb1040eadb3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:04.227459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652a368-be88-4571-88a8-c0a55b8a6a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.227871 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652a368-be88-4571-88a8-c0a55b8a6a81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.228860 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652a368-be88-4571-88a8-c0a55b8a6a81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:04.231149 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652a368-be88-4571-88a8-c0a55b8a6a81 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:04.259493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15fffb-3f69-4754-a453-96e35506820f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:04.260936 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15fffb-3f69-4754-a453-96e35506820f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.261277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15fffb-3f69-4754-a453-96e35506820f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.261521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15fffb-3f69-4754-a453-96e35506820f admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.262279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15fffb-3f69-4754-a453-96e35506820f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:04.262591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15fffb-3f69-4754-a453-96e35506820f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:04.268214 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15fffb-3f69-4754-a453-96e35506820f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:04.312743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 772/1545] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:04.314342 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3219f7a-d26d-4f14-8a1e-70dadda978c0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.314698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3219f7a-d26d-4f14-8a1e-70dadda978c0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.314991 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3219f7a-d26d-4f14-8a1e-70dadda978c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.356924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b3219f7a-d26d-4f14-8a1e-70dadda978c0 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:04.358284 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b3219f7a-d26d-4f14-8a1e-70dadda978c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:04.375132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 773/1546] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:04.375806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3219f7a-d26d-4f14-8a1e-70dadda978c0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:04.391042 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3219f7a-d26d-4f14-8a1e-70dadda978c0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:04.402592 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bac2be-b943-415d-9b4b-ad9d7a1d55a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:04.409515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bac2be-b943-415d-9b4b-ad9d7a1d55a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.409515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bac2be-b943-415d-9b4b-ad9d7a1d55a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.409515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bac2be-b943-415d-9b4b-ad9d7a1d55a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.409515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bac2be-b943-415d-9b4b-ad9d7a1d55a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:04.409515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-40bac2be-b943-415d-9b4b-ad9d7a1d55a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:38:04.409515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 07 08:38:04.409515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: Jun 07 08:38:04.410340 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bac2be-b943-415d-9b4b-ad9d7a1d55a4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:04.410430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 774/1547] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:04 2022] POST /identity/v3/projects => generated 391 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:04.410430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fc7e0-534f-48d4-8be7-7eb7e2ee0d0b None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.410592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fc7e0-534f-48d4-8be7-7eb7e2ee0d0b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.411382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fc7e0-534f-48d4-8be7-7eb7e2ee0d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.459753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7e1fc7e0-534f-48d4-8be7-7eb7e2ee0d0b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:04.459753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7e1fc7e0-534f-48d4-8be7-7eb7e2ee0d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:04.571263 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 774/1548] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:04 2022] POST /identity/v3/projects => generated 383 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:04.571726 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 775/1549] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:04.572858 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fc7e0-534f-48d4-8be7-7eb7e2ee0d0b None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:04.575544 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c46c2-1f46-4da8-a094-d1df3d7bd284 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.575927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c46c2-1f46-4da8-a094-d1df3d7bd284 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.580297 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c46c2-1f46-4da8-a094-d1df3d7bd284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.595416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fc7e0-534f-48d4-8be7-7eb7e2ee0d0b None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:04.612128 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6d3c46c2-1f46-4da8-a094-d1df3d7bd284 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:04.613526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6d3c46c2-1f46-4da8-a094-d1df3d7bd284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:04.710491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a572c-0c97-4616-b714-9edfe86fb06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:04.711086 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a572c-0c97-4616-b714-9edfe86fb06e admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.711400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a572c-0c97-4616-b714-9edfe86fb06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.711681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a572c-0c97-4616-b714-9edfe86fb06e admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.712398 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a572c-0c97-4616-b714-9edfe86fb06e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:04.713489 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: WARNING py.warnings [None req-8c8a572c-0c97-4616-b714-9edfe86fb06e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:38:04.713489 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: warnings.warn(msg) Jun 07 08:38:04.713489 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: Jun 07 08:38:04.714753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a572c-0c97-4616-b714-9edfe86fb06e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:04.749891 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 775/1550] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:04.751998 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ba93f-56ca-4512-902b-e91b735aab1a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.752531 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ba93f-56ca-4512-902b-e91b735aab1a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.753063 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ba93f-56ca-4512-902b-e91b735aab1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.771912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-725ba93f-56ca-4512-902b-e91b735aab1a None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:04.773643 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-725ba93f-56ca-4512-902b-e91b735aab1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:04.781683 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 776/1551] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:04.782539 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ba93f-56ca-4512-902b-e91b735aab1a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:04.793392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ba93f-56ca-4512-902b-e91b735aab1a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:04.798123 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35bd9d-ce34-4356-bb19-2022e8755f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:04.803708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35bd9d-ce34-4356-bb19-2022e8755f6f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.803708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35bd9d-ce34-4356-bb19-2022e8755f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.803708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35bd9d-ce34-4356-bb19-2022e8755f6f admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.803708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35bd9d-ce34-4356-bb19-2022e8755f6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:04.803708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35bd9d-ce34-4356-bb19-2022e8755f6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:04.803708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35bd9d-ce34-4356-bb19-2022e8755f6f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:04.816267 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 777/1552] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:04.817221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35bd9d-ce34-4356-bb19-2022e8755f6f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:04.824820 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 776/1553] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:04 2022] POST /identity/v3/users => generated 296 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:04.828226 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35bd9d-ce34-4356-bb19-2022e8755f6f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:04.828933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67a505-7447-48ae-bb16-9cfd380bcfcf None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.829371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67a505-7447-48ae-bb16-9cfd380bcfcf None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.829761 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67a505-7447-48ae-bb16-9cfd380bcfcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.833590 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2f155-ed8d-4a7e-a6ee-14d0fa62f8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:04.834281 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2f155-ed8d-4a7e-a6ee-14d0fa62f8ad admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.834565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2f155-ed8d-4a7e-a6ee-14d0fa62f8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.834841 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2f155-ed8d-4a7e-a6ee-14d0fa62f8ad admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.835673 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a2f155-ed8d-4a7e-a6ee-14d0fa62f8ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:04.837742 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a2f155-ed8d-4a7e-a6ee-14d0fa62f8ad admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:04.850613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-2d67a505-7447-48ae-bb16-9cfd380bcfcf None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:04.852347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2d67a505-7447-48ae-bb16-9cfd380bcfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:04.861177 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 777/1554] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:04.862574 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aa1be-6c4b-420c-8793-3973a23654e5 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.862874 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aa1be-6c4b-420c-8793-3973a23654e5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.863308 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aa1be-6c4b-420c-8793-3973a23654e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.884608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-fa5aa1be-6c4b-420c-8793-3973a23654e5 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:04.885883 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-fa5aa1be-6c4b-420c-8793-3973a23654e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:04.894323 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 778/1555] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:04.901663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3eb4ed-0eb5-47b0-a68f-5f33849201bd None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.901663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3eb4ed-0eb5-47b0-a68f-5f33849201bd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.901663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3eb4ed-0eb5-47b0-a68f-5f33849201bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.919998 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-8c3eb4ed-0eb5-47b0-a68f-5f33849201bd None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:04.920434 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 778/1556] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:04 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:04.926436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8c3eb4ed-0eb5-47b0-a68f-5f33849201bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:04.926604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e26c2-4042-4758-bd8a-4ff0fc4b2df5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.926604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e26c2-4042-4758-bd8a-4ff0fc4b2df5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.926604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e26c2-4042-4758-bd8a-4ff0fc4b2df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.931117 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 779/1557] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:04.933242 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7402f3f-6254-499d-bb9f-5fa758750e92 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.933571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7402f3f-6254-499d-bb9f-5fa758750e92 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.933951 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7402f3f-6254-499d-bb9f-5fa758750e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.943395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-681e26c2-4042-4758-bd8a-4ff0fc4b2df5 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:04.944703 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-681e26c2-4042-4758-bd8a-4ff0fc4b2df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:04.953251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 779/1558] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:04.955061 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e26c2-4042-4758-bd8a-4ff0fc4b2df5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:04.955194 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d7402f3f-6254-499d-bb9f-5fa758750e92 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:04.962308 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d7402f3f-6254-499d-bb9f-5fa758750e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:04.966077 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e26c2-4042-4758-bd8a-4ff0fc4b2df5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:04.970574 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 780/1559] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:04.971108 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41540af2-5893-455c-8a55-8f44e0bc623c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:04.971553 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7402f3f-6254-499d-bb9f-5fa758750e92 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:04.972251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41540af2-5893-455c-8a55-8f44e0bc623c admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.972676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41540af2-5893-455c-8a55-8f44e0bc623c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.973132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41540af2-5893-455c-8a55-8f44e0bc623c admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.974855 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41540af2-5893-455c-8a55-8f44e0bc623c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:04.977470 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41540af2-5893-455c-8a55-8f44e0bc623c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:04.983623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7402f3f-6254-499d-bb9f-5fa758750e92 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:04.988684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a78fb-fa3c-47e6-b580-27ee6222e687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:04.989006 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a78fb-fa3c-47e6-b580-27ee6222e687 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:04.989304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a78fb-fa3c-47e6-b580-27ee6222e687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:04.989750 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a78fb-fa3c-47e6-b580-27ee6222e687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:04.991499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a78fb-fa3c-47e6-b580-27ee6222e687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:04.995144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a78fb-fa3c-47e6-b580-27ee6222e687 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.009884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 781/1560] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:05.016274 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69d789-04c4-4e57-9d33-d606dcd9fb58 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.016274 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69d789-04c4-4e57-9d33-d606dcd9fb58 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.016274 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69d789-04c4-4e57-9d33-d606dcd9fb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.031694 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7d69d789-04c4-4e57-9d33-d606dcd9fb58 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:05.032926 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7d69d789-04c4-4e57-9d33-d606dcd9fb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:05.048184 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 782/1561] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:05.049631 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8def99-022d-4225-bbe6-d0185aa075fd None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.049918 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8def99-022d-4225-bbe6-d0185aa075fd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.050196 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8def99-022d-4225-bbe6-d0185aa075fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.058047 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:05.069975 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-0c8def99-022d-4225-bbe6-d0185aa075fd None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:05.073457 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0c8def99-022d-4225-bbe6-d0185aa075fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:05.079361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 783/1562] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:05.080203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8def99-022d-4225-bbe6-d0185aa075fd None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:05.090539 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8def99-022d-4225-bbe6-d0185aa075fd None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:05.095282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8dcb6-0636-4626-b8a8-35edd80cf703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:05.095958 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8dcb6-0636-4626-b8a8-35edd80cf703 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.096242 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8dcb6-0636-4626-b8a8-35edd80cf703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.096519 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8dcb6-0636-4626-b8a8-35edd80cf703 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.097289 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8dcb6-0636-4626-b8a8-35edd80cf703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:05.098906 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8dcb6-0636-4626-b8a8-35edd80cf703 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.104108 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 780/1563] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:04 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:05.104108 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41540af2-5893-455c-8a55-8f44e0bc623c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:05.114512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 784/1564] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:05.116391 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbebc6-1781-44f8-9890-e011002ae535 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.119040 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbebc6-1781-44f8-9890-e011002ae535 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.119040 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbebc6-1781-44f8-9890-e011002ae535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.136876 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-08dbebc6-1781-44f8-9890-e011002ae535 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:05.140271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-08dbebc6-1781-44f8-9890-e011002ae535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:05.151644 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:05.184335 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41540af2-5893-455c-8a55-8f44e0bc623c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:05.192123 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd18f1c-3578-4f09-8f78-c3a93e809df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:05.192123 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd18f1c-3578-4f09-8f78-c3a93e809df2 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.192123 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd18f1c-3578-4f09-8f78-c3a93e809df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.199350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd18f1c-3578-4f09-8f78-c3a93e809df2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.199350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd18f1c-3578-4f09-8f78-c3a93e809df2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:05.199350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd18f1c-3578-4f09-8f78-c3a93e809df2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:05.199350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd18f1c-3578-4f09-8f78-c3a93e809df2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.208653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 781/1565] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:05.216723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd18f1c-3578-4f09-8f78-c3a93e809df2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:05.222830 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd18f1c-3578-4f09-8f78-c3a93e809df2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:05.228784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15050cf-5167-46be-8bd2-1cb18d3fa25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:05.229493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15050cf-5167-46be-8bd2-1cb18d3fa25e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.229785 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15050cf-5167-46be-8bd2-1cb18d3fa25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.230062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15050cf-5167-46be-8bd2-1cb18d3fa25e admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.230790 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15050cf-5167-46be-8bd2-1cb18d3fa25e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:05.231100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15050cf-5167-46be-8bd2-1cb18d3fa25e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:05.231206 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 785/1566] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:05.232108 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbebc6-1781-44f8-9890-e011002ae535 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:05.232793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15050cf-5167-46be-8bd2-1cb18d3fa25e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.243009 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbebc6-1781-44f8-9890-e011002ae535 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:05.247771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fab682-d5cf-40c0-8f5f-cc5bedf90c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:05.248455 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fab682-d5cf-40c0-8f5f-cc5bedf90c8d admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.248832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fab682-d5cf-40c0-8f5f-cc5bedf90c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.249146 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fab682-d5cf-40c0-8f5f-cc5bedf90c8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.249389 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 782/1567] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:05.257823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fab682-d5cf-40c0-8f5f-cc5bedf90c8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:05.257994 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15050cf-5167-46be-8bd2-1cb18d3fa25e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:05.258160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fab682-d5cf-40c0-8f5f-cc5bedf90c8d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.261530 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15050cf-5167-46be-8bd2-1cb18d3fa25e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:05.267354 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 786/1568] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:05.267591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33de20-53e1-4f3d-b753-f81bcef28c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:05.268164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33de20-53e1-4f3d-b753-f81bcef28c7d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.268337 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fab682-d5cf-40c0-8f5f-cc5bedf90c8d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:05.268508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33de20-53e1-4f3d-b753-f81bcef28c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.268731 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33de20-53e1-4f3d-b753-f81bcef28c7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.269495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33de20-53e1-4f3d-b753-f81bcef28c7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:05.269805 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33de20-53e1-4f3d-b753-f81bcef28c7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:05.271657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33de20-53e1-4f3d-b753-f81bcef28c7d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.279412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fab682-d5cf-40c0-8f5f-cc5bedf90c8d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:05.284983 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b87a65-5613-4b57-82de-f954f54e8802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:05.285200 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 783/1569] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:05.285757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b87a65-5613-4b57-82de-f954f54e8802 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.286048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b87a65-5613-4b57-82de-f954f54e8802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.286175 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33de20-53e1-4f3d-b753-f81bcef28c7d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:05.286308 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b87a65-5613-4b57-82de-f954f54e8802 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.287042 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b87a65-5613-4b57-82de-f954f54e8802 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:05.287345 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b87a65-5613-4b57-82de-f954f54e8802 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:05.289598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b87a65-5613-4b57-82de-f954f54e8802 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.303201 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33de20-53e1-4f3d-b753-f81bcef28c7d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:05.304084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 787/1570] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:05.304949 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b87a65-5613-4b57-82de-f954f54e8802 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:05.308512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29be74-4f83-4de5-9a6d-d9af2136b091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:05.309252 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29be74-4f83-4de5-9a6d-d9af2136b091 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.309528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29be74-4f83-4de5-9a6d-d9af2136b091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.309802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29be74-4f83-4de5-9a6d-d9af2136b091 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.310499 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29be74-4f83-4de5-9a6d-d9af2136b091 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:05.312743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29be74-4f83-4de5-9a6d-d9af2136b091 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.317385 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b87a65-5613-4b57-82de-f954f54e8802 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:05.325087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb5d1c-7710-48fa-acd5-970207d92b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:05.325841 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb5d1c-7710-48fa-acd5-970207d92b86 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.326165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb5d1c-7710-48fa-acd5-970207d92b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.326495 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb5d1c-7710-48fa-acd5-970207d92b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.327289 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb5d1c-7710-48fa-acd5-970207d92b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:05.329623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb5d1c-7710-48fa-acd5-970207d92b86 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.340501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 788/1571] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:05.341547 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb5d1c-7710-48fa-acd5-970207d92b86 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:05.352399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb5d1c-7710-48fa-acd5-970207d92b86 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:05.358654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a0032-0932-4c5d-a96b-00df66d67962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:05.359765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a0032-0932-4c5d-a96b-00df66d67962 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.360278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a0032-0932-4c5d-a96b-00df66d67962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.360735 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a0032-0932-4c5d-a96b-00df66d67962 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.361534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a0032-0932-4c5d-a96b-00df66d67962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:05.364630 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a0032-0932-4c5d-a96b-00df66d67962 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.399324 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 784/1572] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:05 2022] POST /identity/v3/users => generated 290 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:05.400232 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29be74-4f83-4de5-9a6d-d9af2136b091 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:05.410457 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29be74-4f83-4de5-9a6d-d9af2136b091 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:05.415178 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531135a-017f-4dac-ace3-ea1a7a32e7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:05.415843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531135a-017f-4dac-ace3-ea1a7a32e7c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.416128 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531135a-017f-4dac-ace3-ea1a7a32e7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.416409 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531135a-017f-4dac-ace3-ea1a7a32e7c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.417169 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531135a-017f-4dac-ace3-ea1a7a32e7c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:05.417479 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531135a-017f-4dac-ace3-ea1a7a32e7c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:05.419114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531135a-017f-4dac-ace3-ea1a7a32e7c0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.432647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 785/1573] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:05.437180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531135a-017f-4dac-ace3-ea1a7a32e7c0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:05.443923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531135a-017f-4dac-ace3-ea1a7a32e7c0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:05.449550 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c809294-97f9-457f-a660-429aca24b90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:05.450205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c809294-97f9-457f-a660-429aca24b90b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.450484 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c809294-97f9-457f-a660-429aca24b90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.450758 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c809294-97f9-457f-a660-429aca24b90b admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.451458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c809294-97f9-457f-a660-429aca24b90b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:05.453419 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c809294-97f9-457f-a660-429aca24b90b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.465319 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 789/1574] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:05 2022] POST /identity/v3/projects => generated 389 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:05.465951 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a0032-0932-4c5d-a96b-00df66d67962 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:05.544948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a0032-0932-4c5d-a96b-00df66d67962 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:05.549851 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7310158-c720-4a76-b16c-1c2f58b3f67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:05.550746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7310158-c720-4a76-b16c-1c2f58b3f67c admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.551028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7310158-c720-4a76-b16c-1c2f58b3f67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.551310 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7310158-c720-4a76-b16c-1c2f58b3f67c admin admin] PATH_INFO: `/v3/projects/10b79d91933b4f7f9f32d66138e18373/users/055ec8a54bdf400fac0a46b0e5497f1b/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.560404 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 786/1575] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:05 2022] POST /identity/v3/projects => generated 403 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:05.561428 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c809294-97f9-457f-a660-429aca24b90b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:05.568827 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7310158-c720-4a76-b16c-1c2f58b3f67c admin admin] RBAC: Authorizing `identity:create_grant(project_id=10b79d91933b4f7f9f32d66138e18373, user_id=055ec8a54bdf400fac0a46b0e5497f1b, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:05.572091 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7310158-c720-4a76-b16c-1c2f58b3f67c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.626187 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c809294-97f9-457f-a660-429aca24b90b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:05.630927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2654c-f70c-4621-a260-4ccf924ba0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:05.631597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2654c-f70c-4621-a260-4ccf924ba0bd admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.631878 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2654c-f70c-4621-a260-4ccf924ba0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.632645 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2654c-f70c-4621-a260-4ccf924ba0bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.633411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2654c-f70c-4621-a260-4ccf924ba0bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:05.635333 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2654c-f70c-4621-a260-4ccf924ba0bd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.676588 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 790/1576] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:05 2022] PUT /identity/v3/projects/10b79d91933b4f7f9f32d66138e18373/users/055ec8a54bdf400fac0a46b0e5497f1b/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:05.677516 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7310158-c720-4a76-b16c-1c2f58b3f67c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:05.691113 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:05.743419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7310158-c720-4a76-b16c-1c2f58b3f67c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:05.748149 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a08ef1-c4da-41e7-b4c8-158ced2f54fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:05.748806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a08ef1-c4da-41e7-b4c8-158ced2f54fb admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.749121 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a08ef1-c4da-41e7-b4c8-158ced2f54fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.749395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a08ef1-c4da-41e7-b4c8-158ced2f54fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.750086 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a08ef1-c4da-41e7-b4c8-158ced2f54fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:05.752027 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a08ef1-c4da-41e7-b4c8-158ced2f54fb admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.754429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 787/1577] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:05 2022] POST /identity/v3/projects => generated 371 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:05.755256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2654c-f70c-4621-a260-4ccf924ba0bd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:05.793677 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:05.832617 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2654c-f70c-4621-a260-4ccf924ba0bd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:05.840100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca06cfe-f33e-41ba-8004-25bcd4e393f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:05.841010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca06cfe-f33e-41ba-8004-25bcd4e393f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.841302 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca06cfe-f33e-41ba-8004-25bcd4e393f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.841570 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca06cfe-f33e-41ba-8004-25bcd4e393f6 admin admin] PATH_INFO: `/v3/projects/f0e5af0d10f448bdad3536aded64709a/users/89b9c8a15aee4129a24abd82eaa54c05/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.851367 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 791/1578] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:05 2022] POST /identity/v3/projects => generated 355 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:05.852295 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a08ef1-c4da-41e7-b4c8-158ced2f54fb admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:05.861245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca06cfe-f33e-41ba-8004-25bcd4e393f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e5af0d10f448bdad3536aded64709a, user_id=89b9c8a15aee4129a24abd82eaa54c05, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:05.861245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-fca06cfe-f33e-41ba-8004-25bcd4e393f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:38:05.861245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 07 08:38:05.861245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: Jun 07 08:38:05.863332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca06cfe-f33e-41ba-8004-25bcd4e393f6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.925007 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a08ef1-c4da-41e7-b4c8-158ced2f54fb admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:05.930593 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be174d-05c5-471a-b071-d7a0b54213bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:05.931275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be174d-05c5-471a-b071-d7a0b54213bd admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.931558 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be174d-05c5-471a-b071-d7a0b54213bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.931843 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be174d-05c5-471a-b071-d7a0b54213bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.932580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be174d-05c5-471a-b071-d7a0b54213bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:05.934323 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be174d-05c5-471a-b071-d7a0b54213bd admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.945545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 792/1579] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:05.954456 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be174d-05c5-471a-b071-d7a0b54213bd admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:05.956897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be174d-05c5-471a-b071-d7a0b54213bd admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:05.961748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddcb41-6f41-40a4-89f9-a0be41329f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:05.962409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddcb41-6f41-40a4-89f9-a0be41329f92 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:05.962695 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddcb41-6f41-40a4-89f9-a0be41329f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:05.962978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddcb41-6f41-40a4-89f9-a0be41329f92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:05.963699 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddcb41-6f41-40a4-89f9-a0be41329f92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:05.965674 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddcb41-6f41-40a4-89f9-a0be41329f92 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:05.993471 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 788/1580] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:05 2022] PUT /identity/v3/projects/f0e5af0d10f448bdad3536aded64709a/users/89b9c8a15aee4129a24abd82eaa54c05/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:05.994316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca06cfe-f33e-41ba-8004-25bcd4e393f6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.066495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca06cfe-f33e-41ba-8004-25bcd4e393f6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.068770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 793/1581] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:05 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:06.069802 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddcb41-6f41-40a4-89f9-a0be41329f92 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.135591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ae107-d1d9-437a-ab70-d3a458f3dde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.136328 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ae107-d1d9-437a-ab70-d3a458f3dde1 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.136917 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ae107-d1d9-437a-ab70-d3a458f3dde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.137374 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ae107-d1d9-437a-ab70-d3a458f3dde1 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.138307 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ae107-d1d9-437a-ab70-d3a458f3dde1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.140486 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ae107-d1d9-437a-ab70-d3a458f3dde1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.140980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddcb41-6f41-40a4-89f9-a0be41329f92 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.145497 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49686139-c7ad-4c38-8c89-c9a6e6fa8b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.148868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49686139-c7ad-4c38-8c89-c9a6e6fa8b82 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.148868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49686139-c7ad-4c38-8c89-c9a6e6fa8b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.148868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49686139-c7ad-4c38-8c89-c9a6e6fa8b82 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.148868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49686139-c7ad-4c38-8c89-c9a6e6fa8b82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.149440 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49686139-c7ad-4c38-8c89-c9a6e6fa8b82 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.256718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 789/1582] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:05 2022] POST /identity/v3/users => generated 295 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:06.258138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a6e8a-37e6-4b26-8cd0-5947f62aa4a8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.258420 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a6e8a-37e6-4b26-8cd0-5947f62aa4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.258693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a6e8a-37e6-4b26-8cd0-5947f62aa4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.273222 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 794/1583] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:06 2022] POST /identity/v3/users => generated 302 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:06.274307 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49686139-c7ad-4c38-8c89-c9a6e6fa8b82 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.278643 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c56a6e8a-37e6-4b26-8cd0-5947f62aa4a8 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:06.281784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c56a6e8a-37e6-4b26-8cd0-5947f62aa4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:06.286720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:06.292493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 790/1584] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:06.294489 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a6e8a-37e6-4b26-8cd0-5947f62aa4a8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.294607 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49686139-c7ad-4c38-8c89-c9a6e6fa8b82 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.305174 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a6e8a-37e6-4b26-8cd0-5947f62aa4a8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.307068 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d249962-1e23-47b6-96b2-17d8e3e0b8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.307756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d249962-1e23-47b6-96b2-17d8e3e0b8f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.308094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d249962-1e23-47b6-96b2-17d8e3e0b8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.308421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d249962-1e23-47b6-96b2-17d8e3e0b8f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.309226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d249962-1e23-47b6-96b2-17d8e3e0b8f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.310679 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b8e23-c0d1-43c3-ae55-946f161f56f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.311358 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d249962-1e23-47b6-96b2-17d8e3e0b8f9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.311896 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b8e23-c0d1-43c3-ae55-946f161f56f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.316317 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b8e23-c0d1-43c3-ae55-946f161f56f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.316317 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b8e23-c0d1-43c3-ae55-946f161f56f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.316317 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b8e23-c0d1-43c3-ae55-946f161f56f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.316779 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b8e23-c0d1-43c3-ae55-946f161f56f3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.407855 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 795/1585] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:06 2022] POST /identity/v3/users => generated 286 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:06.408061 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 791/1586] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:06 2022] POST /identity/v3/users => generated 278 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:06.408167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d249962-1e23-47b6-96b2-17d8e3e0b8f9 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.408258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c9e0c-78ab-4168-9ec6-8a43fa5edb9e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.408258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c9e0c-78ab-4168-9ec6-8a43fa5edb9e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.408258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c9e0c-78ab-4168-9ec6-8a43fa5edb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.418705 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d249962-1e23-47b6-96b2-17d8e3e0b8f9 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.424886 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba77aba-19f7-4d49-b400-f2923b324cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.425762 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba77aba-19f7-4d49-b400-f2923b324cdd admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.431328 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba77aba-19f7-4d49-b400-f2923b324cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.431328 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba77aba-19f7-4d49-b400-f2923b324cdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.431328 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba77aba-19f7-4d49-b400-f2923b324cdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.431328 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba77aba-19f7-4d49-b400-f2923b324cdd admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.436298 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f24c9e0c-78ab-4168-9ec6-8a43fa5edb9e None None] MFA Rules not processed for user `89b9c8a15aee4129a24abd82eaa54c05`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:06.437669 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f24c9e0c-78ab-4168-9ec6-8a43fa5edb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:06.440829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 796/1587] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:06.441689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba77aba-19f7-4d49-b400-f2923b324cdd admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.451761 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba77aba-19f7-4d49-b400-f2923b324cdd admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.456482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738f93b-82af-4cc6-a65f-86ddc99d3b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.457183 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738f93b-82af-4cc6-a65f-86ddc99d3b0d admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.457322 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:06.457501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738f93b-82af-4cc6-a65f-86ddc99d3b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.457806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738f93b-82af-4cc6-a65f-86ddc99d3b0d admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.458489 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2738f93b-82af-4cc6-a65f-86ddc99d3b0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.460643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2738f93b-82af-4cc6-a65f-86ddc99d3b0d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.514338 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 792/1588] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:06.515390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c9e0c-78ab-4168-9ec6-8a43fa5edb9e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.526409 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c9e0c-78ab-4168-9ec6-8a43fa5edb9e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.531430 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53a921-f4c1-4919-94c2-1bb8baea7c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.532107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53a921-f4c1-4919-94c2-1bb8baea7c53 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.532397 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53a921-f4c1-4919-94c2-1bb8baea7c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.532787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53a921-f4c1-4919-94c2-1bb8baea7c53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.533607 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53a921-f4c1-4919-94c2-1bb8baea7c53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.535546 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53a921-f4c1-4919-94c2-1bb8baea7c53 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.544671 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 797/1589] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:06 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:06.557762 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738f93b-82af-4cc6-a65f-86ddc99d3b0d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.557920 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 793/1590] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:06.557920 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53a921-f4c1-4919-94c2-1bb8baea7c53 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.561866 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738f93b-82af-4cc6-a65f-86ddc99d3b0d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.567326 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f1770-5a6c-4cfb-8709-0992627fbfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.568000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f1770-5a6c-4cfb-8709-0992627fbfa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.568286 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f1770-5a6c-4cfb-8709-0992627fbfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.568586 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f1770-5a6c-4cfb-8709-0992627fbfa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.569375 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f1770-5a6c-4cfb-8709-0992627fbfa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.571054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f1770-5a6c-4cfb-8709-0992627fbfa0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.581598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 798/1591] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:06.582445 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f1770-5a6c-4cfb-8709-0992627fbfa0 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.593203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f1770-5a6c-4cfb-8709-0992627fbfa0 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.604103 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00c46b-702d-4fd0-b58e-5588272c87f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.604874 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00c46b-702d-4fd0-b58e-5588272c87f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.605247 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00c46b-702d-4fd0-b58e-5588272c87f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.605851 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00c46b-702d-4fd0-b58e-5588272c87f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.606661 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00c46b-702d-4fd0-b58e-5588272c87f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.608568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00c46b-702d-4fd0-b58e-5588272c87f4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.620707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53a921-f4c1-4919-94c2-1bb8baea7c53 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.623251 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 799/1592] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:06.624347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00c46b-702d-4fd0-b58e-5588272c87f4 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.626981 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7213e-e1c9-4e4b-b483-55d5bbfe2008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.627627 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7213e-e1c9-4e4b-b483-55d5bbfe2008 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.627903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7213e-e1c9-4e4b-b483-55d5bbfe2008 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.628229 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7213e-e1c9-4e4b-b483-55d5bbfe2008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.634590 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00c46b-702d-4fd0-b58e-5588272c87f4 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.638006 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7213e-e1c9-4e4b-b483-55d5bbfe2008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.640824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7213e-e1c9-4e4b-b483-55d5bbfe2008 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.640954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9bd09-e56f-44f9-9fc0-6a322c66e3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.640954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9bd09-e56f-44f9-9fc0-6a322c66e3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.640954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9bd09-e56f-44f9-9fc0-6a322c66e3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.640954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9bd09-e56f-44f9-9fc0-6a322c66e3c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.641740 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9bd09-e56f-44f9-9fc0-6a322c66e3c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.643810 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9bd09-e56f-44f9-9fc0-6a322c66e3c8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.649854 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 794/1593] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:06.650717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7213e-e1c9-4e4b-b483-55d5bbfe2008 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.658639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 800/1594] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:06.658639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b785f33-91e9-4f26-9bb8-537cbfb58905 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.658639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b785f33-91e9-4f26-9bb8-537cbfb58905 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.658639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b785f33-91e9-4f26-9bb8-537cbfb58905 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.659089 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 801/1595] 10.222.0.17 () {54 vars in 961 bytes} [Tue Jun 7 08:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:38:06.665778 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7213e-e1c9-4e4b-b483-55d5bbfe2008 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.669770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b785f33-91e9-4f26-9bb8-537cbfb58905 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.671308 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a18b6f-4c2f-4fb3-b037-e9f6a1700285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.672293 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a18b6f-4c2f-4fb3-b037-e9f6a1700285 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.681478 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a18b6f-4c2f-4fb3-b037-e9f6a1700285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.681815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b785f33-91e9-4f26-9bb8-537cbfb58905 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.681989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a18b6f-4c2f-4fb3-b037-e9f6a1700285 admin admin] PATH_INFO: `/v3/projects/f26ee9298ae04230979b6a62d24be0e1/users/ae58168dadc845daa0958cf975a41abd/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.687459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acef31b-dae1-43ce-90ca-90d0bc5469f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.689319 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acef31b-dae1-43ce-90ca-90d0bc5469f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.692743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acef31b-dae1-43ce-90ca-90d0bc5469f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.692743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acef31b-dae1-43ce-90ca-90d0bc5469f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.692743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acef31b-dae1-43ce-90ca-90d0bc5469f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.692743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acef31b-dae1-43ce-90ca-90d0bc5469f2 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.698985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a18b6f-4c2f-4fb3-b037-e9f6a1700285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f26ee9298ae04230979b6a62d24be0e1, user_id=ae58168dadc845daa0958cf975a41abd, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.701764 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a18b6f-4c2f-4fb3-b037-e9f6a1700285 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.702720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 802/1596] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:06.703566 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acef31b-dae1-43ce-90ca-90d0bc5469f2 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.716303 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acef31b-dae1-43ce-90ca-90d0bc5469f2 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.724785 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e2d47-7d0a-4d93-9edb-db03af81a833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.725566 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e2d47-7d0a-4d93-9edb-db03af81a833 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.725853 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e2d47-7d0a-4d93-9edb-db03af81a833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.726808 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e2d47-7d0a-4d93-9edb-db03af81a833 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.727546 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e2d47-7d0a-4d93-9edb-db03af81a833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.729197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e2d47-7d0a-4d93-9edb-db03af81a833 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.742227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 803/1597] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:06.747438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e2d47-7d0a-4d93-9edb-db03af81a833 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.764881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e2d47-7d0a-4d93-9edb-db03af81a833 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.773902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e265a32-e09f-4aab-a05b-10f8daf055a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.774796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e265a32-e09f-4aab-a05b-10f8daf055a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.775248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e265a32-e09f-4aab-a05b-10f8daf055a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.775845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e265a32-e09f-4aab-a05b-10f8daf055a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.776921 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e265a32-e09f-4aab-a05b-10f8daf055a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.779045 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e265a32-e09f-4aab-a05b-10f8daf055a5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.785169 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 795/1598] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:06 2022] PUT /identity/v3/projects/f26ee9298ae04230979b6a62d24be0e1/users/ae58168dadc845daa0958cf975a41abd/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:06.785169 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a18b6f-4c2f-4fb3-b037-e9f6a1700285 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.795154 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 804/1599] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:06.795848 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e265a32-e09f-4aab-a05b-10f8daf055a5 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.806342 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e265a32-e09f-4aab-a05b-10f8daf055a5 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.890401 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d04459-0b45-48a2-8c05-c851d2d75ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.891397 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d04459-0b45-48a2-8c05-c851d2d75ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.891846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d04459-0b45-48a2-8c05-c851d2d75ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.892331 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d04459-0b45-48a2-8c05-c851d2d75ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.893263 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d04459-0b45-48a2-8c05-c851d2d75ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.893725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a18b6f-4c2f-4fb3-b037-e9f6a1700285 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.895683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d04459-0b45-48a2-8c05-c851d2d75ce3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.899400 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab20060-fd7a-45a6-9c78-25d29e88c962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.900152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab20060-fd7a-45a6-9c78-25d29e88c962 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.902739 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab20060-fd7a-45a6-9c78-25d29e88c962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.902739 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab20060-fd7a-45a6-9c78-25d29e88c962 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.902739 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab20060-fd7a-45a6-9c78-25d29e88c962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.903196 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab20060-fd7a-45a6-9c78-25d29e88c962 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.908280 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 805/1600] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:06.910010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439442d-b8ad-4cf6-9427-c38017ba89d1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.910281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439442d-b8ad-4cf6-9427-c38017ba89d1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.910658 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439442d-b8ad-4cf6-9427-c38017ba89d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.919023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 796/1601] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:06.919023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab20060-fd7a-45a6-9c78-25d29e88c962 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.926261 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab20060-fd7a-45a6-9c78-25d29e88c962 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.930975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5068c211-dbb2-4747-a4a3-9795166e9de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.931644 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068c211-dbb2-4747-a4a3-9795166e9de2 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.931986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068c211-dbb2-4747-a4a3-9795166e9de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.932200 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068c211-dbb2-4747-a4a3-9795166e9de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.932331 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d439442d-b8ad-4cf6-9427-c38017ba89d1 None None] MFA Rules not processed for user `ee42568b059b49a198ed2c01d5efdafd`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:06.932918 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5068c211-dbb2-4747-a4a3-9795166e9de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:06.933736 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d439442d-b8ad-4cf6-9427-c38017ba89d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:06.934537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5068c211-dbb2-4747-a4a3-9795166e9de2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:06.951957 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 797/1602] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:06.951957 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5068c211-dbb2-4747-a4a3-9795166e9de2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:06.967253 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5068c211-dbb2-4747-a4a3-9795166e9de2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:06.973600 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e110273-14c2-4677-95f3-032c8244cfc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:06.974678 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e110273-14c2-4677-95f3-032c8244cfc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:06.974855 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e110273-14c2-4677-95f3-032c8244cfc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:06.975133 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e110273-14c2-4677-95f3-032c8244cfc4 admin admin] PATH_INFO: `/v3/projects/4f25f022755d465fac40f2681eaef0a5/users/0b2efb9857cf495f8b51a96b5f868d44/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:06.988658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:06.996243 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:06.997714 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e110273-14c2-4677-95f3-032c8244cfc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f25f022755d465fac40f2681eaef0a5, user_id=0b2efb9857cf495f8b51a96b5f868d44, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:07.001109 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e110273-14c2-4677-95f3-032c8244cfc4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:07.040770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 806/1603] 10.222.0.17 () {58 vars in 1058 bytes} [Tue Jun 7 08:38:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:07.042412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaace31-b10d-4ea8-917e-d898f60e2b2e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.042783 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaace31-b10d-4ea8-917e-d898f60e2b2e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.043186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaace31-b10d-4ea8-917e-d898f60e2b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.062583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9eaace31-b10d-4ea8-917e-d898f60e2b2e None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:07.063810 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9eaace31-b10d-4ea8-917e-d898f60e2b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:07.072473 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 807/1604] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:07.073410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaace31-b10d-4ea8-917e-d898f60e2b2e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:07.084669 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaace31-b10d-4ea8-917e-d898f60e2b2e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:07.090133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2d19e-fd75-417b-8fd9-54926d5f0292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:07.092243 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2d19e-fd75-417b-8fd9-54926d5f0292 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.092243 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2d19e-fd75-417b-8fd9-54926d5f0292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.092243 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2d19e-fd75-417b-8fd9-54926d5f0292 admin admin] PATH_INFO: `/v3/projects/448b9f27e4bc4870aebf03e2f8bca11e/users/6c240d9d8a9a4d1a8bb43c1151edbe38/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.107096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d2d19e-fd75-417b-8fd9-54926d5f0292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=448b9f27e4bc4870aebf03e2f8bca11e, user_id=6c240d9d8a9a4d1a8bb43c1151edbe38, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:07.109774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d2d19e-fd75-417b-8fd9-54926d5f0292 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:07.118064 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 798/1605] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:06 2022] PUT /identity/v3/projects/4f25f022755d465fac40f2681eaef0a5/users/0b2efb9857cf495f8b51a96b5f868d44/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:07.118875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e110273-14c2-4677-95f3-032c8244cfc4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:07.184278 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e110273-14c2-4677-95f3-032c8244cfc4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:07.197408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 808/1606] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:07 2022] PUT /identity/v3/projects/448b9f27e4bc4870aebf03e2f8bca11e/users/6c240d9d8a9a4d1a8bb43c1151edbe38/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:07.197408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2d19e-fd75-417b-8fd9-54926d5f0292 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:07.274164 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2d19e-fd75-417b-8fd9-54926d5f0292 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:07.279311 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8dc1c6-5a56-491d-8f29-65818dacb351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:07.282174 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dc1c6-5a56-491d-8f29-65818dacb351 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.282174 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dc1c6-5a56-491d-8f29-65818dacb351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.282174 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dc1c6-5a56-491d-8f29-65818dacb351 admin admin] PATH_INFO: `/v3/projects/352bab09470a4f67bec618675f8d5fa0/users/8b235adbf9ad4dbda55ea361e5c7fd2b/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.286874 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fe83b-df29-4b57-acdd-274526693163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:07.288253 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fe83b-df29-4b57-acdd-274526693163 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.288710 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fe83b-df29-4b57-acdd-274526693163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.289161 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fe83b-df29-4b57-acdd-274526693163 admin admin] PATH_INFO: `/v3/projects/e4d717cdae144d69ac48fc650e970628/users/49d3620d0d3c4924ba3e15b0974bc357/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.300039 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8dc1c6-5a56-491d-8f29-65818dacb351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=352bab09470a4f67bec618675f8d5fa0, user_id=8b235adbf9ad4dbda55ea361e5c7fd2b, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:07.305340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8dc1c6-5a56-491d-8f29-65818dacb351 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:07.316381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fe83b-df29-4b57-acdd-274526693163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d717cdae144d69ac48fc650e970628, user_id=49d3620d0d3c4924ba3e15b0974bc357, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:07.321627 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fe83b-df29-4b57-acdd-274526693163 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:07.388343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 809/1607] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:07 2022] PUT /identity/v3/projects/352bab09470a4f67bec618675f8d5fa0/users/8b235adbf9ad4dbda55ea361e5c7fd2b/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:07.389752 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8dc1c6-5a56-491d-8f29-65818dacb351 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:07.415810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 799/1608] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:07 2022] PUT /identity/v3/projects/e4d717cdae144d69ac48fc650e970628/users/49d3620d0d3c4924ba3e15b0974bc357/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:07.419259 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fe83b-df29-4b57-acdd-274526693163 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:07.458603 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8dc1c6-5a56-491d-8f29-65818dacb351 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:07.498420 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:07.515548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fe83b-df29-4b57-acdd-274526693163 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:07.521416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb716ca-4425-4330-a0bf-36235121991a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:07.522082 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb716ca-4425-4330-a0bf-36235121991a service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.522447 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb716ca-4425-4330-a0bf-36235121991a service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.522754 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb716ca-4425-4330-a0bf-36235121991a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.523853 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3be6f-a515-4062-830e-a04d1b5ff25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:07.525144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3be6f-a515-4062-830e-a04d1b5ff25d admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.525482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3be6f-a515-4062-830e-a04d1b5ff25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.526448 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3be6f-a515-4062-830e-a04d1b5ff25d admin admin] PATH_INFO: `/v3/projects/b371f1c3f8914cf4898d65538657bb57/users/ac14029f6fdf43269fbadb2a4c65c6d1/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.538185 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:07.542922 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f3be6f-a515-4062-830e-a04d1b5ff25d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b371f1c3f8914cf4898d65538657bb57, user_id=ac14029f6fdf43269fbadb2a4c65c6d1, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:07.546054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f3be6f-a515-4062-830e-a04d1b5ff25d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:07.595703 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb716ca-4425-4330-a0bf-36235121991a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:07.597383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb716ca-4425-4330-a0bf-36235121991a service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:07.607752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 800/1609] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:07.608598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb716ca-4425-4330-a0bf-36235121991a service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:07.609946 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8cb716ca-4425-4330-a0bf-36235121991a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:07.662685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 810/1610] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:07 2022] PUT /identity/v3/projects/b371f1c3f8914cf4898d65538657bb57/users/ac14029f6fdf43269fbadb2a4c65c6d1/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:07.677355 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c5843-3709-4784-a110-733c9ec36061 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.677355 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c5843-3709-4784-a110-733c9ec36061 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.677355 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c5843-3709-4784-a110-733c9ec36061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.691918 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb716ca-4425-4330-a0bf-36235121991a service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:07.696794 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d81c5843-3709-4784-a110-733c9ec36061 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:07.698061 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d81c5843-3709-4784-a110-733c9ec36061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:07.749407 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6516ff-0680-456c-a4c5-cd36b8f1c7fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:07.750037 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6516ff-0680-456c-a4c5-cd36b8f1c7fb service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.750303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6516ff-0680-456c-a4c5-cd36b8f1c7fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.750565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6516ff-0680-456c-a4c5-cd36b8f1c7fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.806811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 811/1611] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:07.808946 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db90c7f-798e-48f7-95af-58d05058ed59 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.809268 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db90c7f-798e-48f7-95af-58d05058ed59 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.809555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db90c7f-798e-48f7-95af-58d05058ed59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.847136 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6516ff-0680-456c-a4c5-cd36b8f1c7fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:07.848984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6516ff-0680-456c-a4c5-cd36b8f1c7fb service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:07.850126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7db90c7f-798e-48f7-95af-58d05058ed59 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:07.851951 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7db90c7f-798e-48f7-95af-58d05058ed59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:07.863262 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 801/1612] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:07.864904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5b137-c284-43b2-b629-41325e7f4f90 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.865324 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5b137-c284-43b2-b629-41325e7f4f90 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.865697 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5b137-c284-43b2-b629-41325e7f4f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.871863 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 812/1613] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:07.871863 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320bbde-45d6-4fa5-8e3f-6c0c7f693f9e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.871863 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320bbde-45d6-4fa5-8e3f-6c0c7f693f9e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.871863 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320bbde-45d6-4fa5-8e3f-6c0c7f693f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.898056 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a7d5b137-c284-43b2-b629-41325e7f4f90 None None] MFA Rules not processed for user `8b235adbf9ad4dbda55ea361e5c7fd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:07.899556 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a7d5b137-c284-43b2-b629-41325e7f4f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:07.907490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-3320bbde-45d6-4fa5-8e3f-6c0c7f693f9e None None] MFA Rules not processed for user `49d3620d0d3c4924ba3e15b0974bc357`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:07.908931 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3320bbde-45d6-4fa5-8e3f-6c0c7f693f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:07.910054 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:07.973824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 802/1614] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:07.975310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625204bc-0cd1-456f-b22c-edf1aca7713d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:07.975609 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625204bc-0cd1-456f-b22c-edf1aca7713d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:07.976440 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625204bc-0cd1-456f-b22c-edf1aca7713d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:07.994308 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 813/1615] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:07 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:07.994308 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320bbde-45d6-4fa5-8e3f-6c0c7f693f9e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:08.009319 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320bbde-45d6-4fa5-8e3f-6c0c7f693f9e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:08.016958 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827231a0-37c0-4abd-9f86-ec41c556df08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:08.017978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827231a0-37c0-4abd-9f86-ec41c556df08 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.018354 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827231a0-37c0-4abd-9f86-ec41c556df08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.018727 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827231a0-37c0-4abd-9f86-ec41c556df08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.019706 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-625204bc-0cd1-456f-b22c-edf1aca7713d None None] MFA Rules not processed for user `ac14029f6fdf43269fbadb2a4c65c6d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:08.021051 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-625204bc-0cd1-456f-b22c-edf1aca7713d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:08.030418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827231a0-37c0-4abd-9f86-ec41c556df08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.032222 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827231a0-37c0-4abd-9f86-ec41c556df08 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:08.043927 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 814/1616] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:08.045485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305aa68-2c7c-4096-abaf-82570b7b8bc6 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.045933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305aa68-2c7c-4096-abaf-82570b7b8bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.046325 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305aa68-2c7c-4096-abaf-82570b7b8bc6 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.047790 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 815/1617] 10.222.0.17 () {54 vars in 912 bytes} [Tue Jun 7 08:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:38:08.053023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2305aa68-2c7c-4096-abaf-82570b7b8bc6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:08.067345 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2305aa68-2c7c-4096-abaf-82570b7b8bc6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:08.075116 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823756c-a2d3-445b-9489-e478ba8f5ceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:08.075813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823756c-a2d3-445b-9489-e478ba8f5ceb service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.076600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823756c-a2d3-445b-9489-e478ba8f5ceb service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.076933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823756c-a2d3-445b-9489-e478ba8f5ceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.089342 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823756c-a2d3-445b-9489-e478ba8f5ceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.091120 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823756c-a2d3-445b-9489-e478ba8f5ceb service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:08.102060 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 816/1618] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:08.103474 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823756c-a2d3-445b-9489-e478ba8f5ceb service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:08.113280 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823756c-a2d3-445b-9489-e478ba8f5ceb service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:08.118691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd033c4-cbdf-49ab-a2bd-5dbfcf23bb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:08.119122 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 803/1619] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:08.120006 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625204bc-0cd1-456f-b22c-edf1aca7713d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:08.120356 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd033c4-cbdf-49ab-a2bd-5dbfcf23bb32 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.120871 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd033c4-cbdf-49ab-a2bd-5dbfcf23bb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.121253 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd033c4-cbdf-49ab-a2bd-5dbfcf23bb32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.122073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd033c4-cbdf-49ab-a2bd-5dbfcf23bb32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.124328 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd033c4-cbdf-49ab-a2bd-5dbfcf23bb32 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:08.136957 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625204bc-0cd1-456f-b22c-edf1aca7713d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:08.146308 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:08.146715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75646855-0bd3-49d1-a3aa-fc80a32a0877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:08.147608 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75646855-0bd3-49d1-a3aa-fc80a32a0877 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.148349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75646855-0bd3-49d1-a3aa-fc80a32a0877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.148680 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75646855-0bd3-49d1-a3aa-fc80a32a0877 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.149482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75646855-0bd3-49d1-a3aa-fc80a32a0877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.151623 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75646855-0bd3-49d1-a3aa-fc80a32a0877 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:08.282356 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 817/1620] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:08 2022] POST /identity/v3/projects => generated 371 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:08.286131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db352a9-7d72-46d5-9afc-1d9fc88461f9 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.286131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db352a9-7d72-46d5-9afc-1d9fc88461f9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.286131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db352a9-7d72-46d5-9afc-1d9fc88461f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.295836 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 804/1621] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:08 2022] POST /identity/v3/projects => generated 355 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:08.297420 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75646855-0bd3-49d1-a3aa-fc80a32a0877 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:08.333839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-4db352a9-7d72-46d5-9afc-1d9fc88461f9 None None] MFA Rules not processed for user `ee42568b059b49a198ed2c01d5efdafd`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:08.335085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4db352a9-7d72-46d5-9afc-1d9fc88461f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:08.417199 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75646855-0bd3-49d1-a3aa-fc80a32a0877 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:08.422349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e1a86-4153-4aaf-9fd3-07544de4a819 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:08.423008 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e1a86-4153-4aaf-9fd3-07544de4a819 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.423961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e1a86-4153-4aaf-9fd3-07544de4a819 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.423961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e1a86-4153-4aaf-9fd3-07544de4a819 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.454546 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 818/1622] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jun 7 08:38:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:08.455654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db352a9-7d72-46d5-9afc-1d9fc88461f9 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:08.473261 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:08.528238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e1a86-4153-4aaf-9fd3-07544de4a819 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.531373 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e1a86-4153-4aaf-9fd3-07544de4a819 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:08.558888 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 805/1623] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:08.567362 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e1a86-4153-4aaf-9fd3-07544de4a819 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:08.581852 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db352a9-7d72-46d5-9afc-1d9fc88461f9 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:08.588320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e1a86-4153-4aaf-9fd3-07544de4a819 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:08.594143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233c487-60b2-4e30-bd9e-96e8d2bbfb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:08.595397 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233c487-60b2-4e30-bd9e-96e8d2bbfb33 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.597386 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233c487-60b2-4e30-bd9e-96e8d2bbfb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.598317 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233c487-60b2-4e30-bd9e-96e8d2bbfb33 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.601964 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233c487-60b2-4e30-bd9e-96e8d2bbfb33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.603459 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a29b47-e040-4cee-9e7c-fe4fa22a88e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:08.604406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a29b47-e040-4cee-9e7c-fe4fa22a88e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.604817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a29b47-e040-4cee-9e7c-fe4fa22a88e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.605150 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a29b47-e040-4cee-9e7c-fe4fa22a88e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.605944 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a29b47-e040-4cee-9e7c-fe4fa22a88e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.608006 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a29b47-e040-4cee-9e7c-fe4fa22a88e7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:08.619988 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233c487-60b2-4e30-bd9e-96e8d2bbfb33 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:08.727553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 806/1624] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:08 2022] POST /identity/v3/users => generated 284 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:08.728938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a29b47-e040-4cee-9e7c-fe4fa22a88e7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:08.736472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 819/1625] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:08 2022] POST /identity/v3/users => generated 292 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:08.740936 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233c487-60b2-4e30-bd9e-96e8d2bbfb33 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:08.741142 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a29b47-e040-4cee-9e7c-fe4fa22a88e7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:08.747454 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea131d-a3a4-4cac-92b5-ecb4772eb5e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:08.748722 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea131d-a3a4-4cac-92b5-ecb4772eb5e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.749095 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea131d-a3a4-4cac-92b5-ecb4772eb5e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.749437 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea131d-a3a4-4cac-92b5-ecb4772eb5e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.778602 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea131d-a3a4-4cac-92b5-ecb4772eb5e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.784204 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea131d-a3a4-4cac-92b5-ecb4772eb5e1 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:08.791055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233c487-60b2-4e30-bd9e-96e8d2bbfb33 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:08.802856 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35558116-b2e2-4892-8d54-42dc2f4fcafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:08.803634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35558116-b2e2-4892-8d54-42dc2f4fcafb admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.810360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35558116-b2e2-4892-8d54-42dc2f4fcafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.810360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35558116-b2e2-4892-8d54-42dc2f4fcafb admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.810360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35558116-b2e2-4892-8d54-42dc2f4fcafb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.810360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35558116-b2e2-4892-8d54-42dc2f4fcafb admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:08.833392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 807/1626] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:08.833392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea131d-a3a4-4cac-92b5-ecb4772eb5e1 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:08.833604 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 820/1627] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:08.838064 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35558116-b2e2-4892-8d54-42dc2f4fcafb admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:08.856320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea131d-a3a4-4cac-92b5-ecb4772eb5e1 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:08.863555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35558116-b2e2-4892-8d54-42dc2f4fcafb admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:08.871257 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db1639-5150-432d-9854-7a81d7fc9fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:08.872187 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db1639-5150-432d-9854-7a81d7fc9fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.872531 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db1639-5150-432d-9854-7a81d7fc9fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.872925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db1639-5150-432d-9854-7a81d7fc9fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.873763 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db1639-5150-432d-9854-7a81d7fc9fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.879299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db1639-5150-432d-9854-7a81d7fc9fa8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:08.884941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f6d78-96d8-42e2-8d8a-72e2ab86ca8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:08.884941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f6d78-96d8-42e2-8d8a-72e2ab86ca8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.884941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f6d78-96d8-42e2-8d8a-72e2ab86ca8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.884941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f6d78-96d8-42e2-8d8a-72e2ab86ca8e admin admin] PATH_INFO: `/v3/projects/ca5799381f704770879b32537fe0c8ae/users/c446abad56be439b8d93e2953f358960/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.903296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f6d78-96d8-42e2-8d8a-72e2ab86ca8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5799381f704770879b32537fe0c8ae, user_id=c446abad56be439b8d93e2953f358960, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.906295 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 808/1628] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:08.906473 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f6d78-96d8-42e2-8d8a-72e2ab86ca8e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:08.910835 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db1639-5150-432d-9854-7a81d7fc9fa8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:08.936431 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db1639-5150-432d-9854-7a81d7fc9fa8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:08.942900 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ffc3c-e707-44b3-b3f4-668fd3d0dd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:08.947845 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ffc3c-e707-44b3-b3f4-668fd3d0dd9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:08.947845 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ffc3c-e707-44b3-b3f4-668fd3d0dd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:08.947845 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ffc3c-e707-44b3-b3f4-668fd3d0dd9a admin admin] PATH_INFO: `/v3/projects/8f614aff98a742f2b2d9305804396efb/users/1723dcdb1aef47bba89fdcc213f091b5/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:08.986906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ffc3c-e707-44b3-b3f4-668fd3d0dd9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f614aff98a742f2b2d9305804396efb, user_id=1723dcdb1aef47bba89fdcc213f091b5, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:08.989655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ffc3c-e707-44b3-b3f4-668fd3d0dd9a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:09.016350 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 821/1629] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:08 2022] PUT /identity/v3/projects/ca5799381f704770879b32537fe0c8ae/users/c446abad56be439b8d93e2953f358960/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:09.020315 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f6d78-96d8-42e2-8d8a-72e2ab86ca8e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:09.081423 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 809/1630] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:08 2022] PUT /identity/v3/projects/8f614aff98a742f2b2d9305804396efb/users/1723dcdb1aef47bba89fdcc213f091b5/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:38:09.085847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ffc3c-e707-44b3-b3f4-668fd3d0dd9a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:09.190985 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:09.228102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f6d78-96d8-42e2-8d8a-72e2ab86ca8e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:09.259208 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ffc3c-e707-44b3-b3f4-668fd3d0dd9a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:09.274351 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d479d4d-8746-476c-bd74-9c16cd7bfff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:09.275102 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d479d4d-8746-476c-bd74-9c16cd7bfff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:09.275385 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d479d4d-8746-476c-bd74-9c16cd7bfff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:09.275745 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d479d4d-8746-476c-bd74-9c16cd7bfff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:09.278215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d479d4d-8746-476c-bd74-9c16cd7bfff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:09.282916 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d479d4d-8746-476c-bd74-9c16cd7bfff7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:09.306803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 810/1631] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:09.311000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d479d4d-8746-476c-bd74-9c16cd7bfff7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:09.331952 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edc356-40e3-4fca-90ec-4d039a89f898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:09.334675 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edc356-40e3-4fca-90ec-4d039a89f898 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:09.335067 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edc356-40e3-4fca-90ec-4d039a89f898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:09.336314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edc356-40e3-4fca-90ec-4d039a89f898 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:09.338256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49edc356-40e3-4fca-90ec-4d039a89f898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:09.344307 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49edc356-40e3-4fca-90ec-4d039a89f898 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:09.346831 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d479d4d-8746-476c-bd74-9c16cd7bfff7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:09.368252 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495cbe45-2609-4e03-b9d0-4a5f90e8d1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:09.373693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cbe45-2609-4e03-b9d0-4a5f90e8d1ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:09.373693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cbe45-2609-4e03-b9d0-4a5f90e8d1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:09.374060 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 822/1632] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:09.374161 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cbe45-2609-4e03-b9d0-4a5f90e8d1ba admin admin] PATH_INFO: `/v3/projects/8f614aff98a742f2b2d9305804396efb/users/1723dcdb1aef47bba89fdcc213f091b5/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:09.375132 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edc356-40e3-4fca-90ec-4d039a89f898 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:09.385681 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495cbe45-2609-4e03-b9d0-4a5f90e8d1ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f614aff98a742f2b2d9305804396efb, user_id=1723dcdb1aef47bba89fdcc213f091b5, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:09.388411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495cbe45-2609-4e03-b9d0-4a5f90e8d1ba admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:09.398518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edc356-40e3-4fca-90ec-4d039a89f898 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:09.403496 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30cefc8-8ee4-4fbe-b971-48ffdb62a26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:09.404407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cefc8-8ee4-4fbe-b971-48ffdb62a26b admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:09.404686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cefc8-8ee4-4fbe-b971-48ffdb62a26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:09.404961 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cefc8-8ee4-4fbe-b971-48ffdb62a26b admin admin] PATH_INFO: `/v3/projects/ca5799381f704770879b32537fe0c8ae/users/c446abad56be439b8d93e2953f358960/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:09.407025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30cefc8-8ee4-4fbe-b971-48ffdb62a26b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5799381f704770879b32537fe0c8ae, user_id=c446abad56be439b8d93e2953f358960, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:09.410126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30cefc8-8ee4-4fbe-b971-48ffdb62a26b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:09.421746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 823/1633] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:09 2022] PUT /identity/v3/projects/ca5799381f704770879b32537fe0c8ae/users/c446abad56be439b8d93e2953f358960/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:09.433459 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 811/1634] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:09 2022] PUT /identity/v3/projects/8f614aff98a742f2b2d9305804396efb/users/1723dcdb1aef47bba89fdcc213f091b5/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:38:09.435469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d18b3-1621-4bae-a789-9d572573079a None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:09.435762 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d18b3-1621-4bae-a789-9d572573079a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:09.436041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d18b3-1621-4bae-a789-9d572573079a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:09.447168 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b87b31-5d79-4d8a-a6f5-d21bec4eaedd None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:09.447435 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b87b31-5d79-4d8a-a6f5-d21bec4eaedd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:09.447721 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b87b31-5d79-4d8a-a6f5-d21bec4eaedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:09.499961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-92b87b31-5d79-4d8a-a6f5-d21bec4eaedd None None] MFA Rules not processed for user `1723dcdb1aef47bba89fdcc213f091b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:09.500287 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a45d18b3-1621-4bae-a789-9d572573079a None None] MFA Rules not processed for user `c446abad56be439b8d93e2953f358960`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:09.502373 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-92b87b31-5d79-4d8a-a6f5-d21bec4eaedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:09.503757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a45d18b3-1621-4bae-a789-9d572573079a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:09.568911 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:09.604123 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 824/1635] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:09 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:09.633695 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 812/1636] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:09 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:10.046397 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d18b3-1621-4bae-a789-9d572573079a None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:10.193654 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b87b31-5d79-4d8a-a6f5-d21bec4eaedd None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:10.218531 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d18b3-1621-4bae-a789-9d572573079a None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:10.230067 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6446597-433f-447d-8cce-53b5dd6fbd48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:10.231251 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6446597-433f-447d-8cce-53b5dd6fbd48 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:10.231694 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6446597-433f-447d-8cce-53b5dd6fbd48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:10.236452 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6446597-433f-447d-8cce-53b5dd6fbd48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:10.281998 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6446597-433f-447d-8cce-53b5dd6fbd48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:10.283453 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6446597-433f-447d-8cce-53b5dd6fbd48 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:10.303699 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:10.317039 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 825/1637] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:10 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:10.318734 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c70391-f876-47e3-a413-0fc8e52bc72f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:10.319048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c70391-f876-47e3-a413-0fc8e52bc72f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:10.319332 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c70391-f876-47e3-a413-0fc8e52bc72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:10.369302 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b87b31-5d79-4d8a-a6f5-d21bec4eaedd None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:10.380982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61cd57-b731-455c-ba1a-a0de91924673 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:10.381709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61cd57-b731-455c-ba1a-a0de91924673 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:10.381991 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61cd57-b731-455c-ba1a-a0de91924673 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:10.382328 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61cd57-b731-455c-ba1a-a0de91924673 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:10.398255 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-69c70391-f876-47e3-a413-0fc8e52bc72f None None] MFA Rules not processed for user `ae58168dadc845daa0958cf975a41abd`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:10.403456 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-69c70391-f876-47e3-a413-0fc8e52bc72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:10.404961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61cd57-b731-455c-ba1a-a0de91924673 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:10.406467 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61cd57-b731-455c-ba1a-a0de91924673 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:10.431945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 813/1638] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:10 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:10.558498 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 826/1639] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:10.569571 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa751fc0-c450-4417-afff-abc2390215c9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:10.570431 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa751fc0-c450-4417-afff-abc2390215c9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:10.570948 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa751fc0-c450-4417-afff-abc2390215c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:10.615879 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fa751fc0-c450-4417-afff-abc2390215c9 None None] MFA Rules not processed for user `ae58168dadc845daa0958cf975a41abd`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:10.621233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fa751fc0-c450-4417-afff-abc2390215c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:10.644671 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 814/1640] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:10.698955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563660f-5b3b-40b1-b4a0-ccbd01cef0e4 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:10.699245 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563660f-5b3b-40b1-b4a0-ccbd01cef0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:10.699531 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563660f-5b3b-40b1-b4a0-ccbd01cef0e4 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:10.701797 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 827/1641] 10.222.0.17 () {54 vars in 961 bytes} [Tue Jun 7 08:38:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:38:10.729763 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea84c2-8a3d-491e-b033-a08b6e28f08c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:10.730000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea84c2-8a3d-491e-b033-a08b6e28f08c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:10.730299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea84c2-8a3d-491e-b033-a08b6e28f08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:10.816649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-caea84c2-8a3d-491e-b033-a08b6e28f08c None None] MFA Rules not processed for user `ee42568b059b49a198ed2c01d5efdafd`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:10.818842 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-caea84c2-8a3d-491e-b033-a08b6e28f08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:10.931238 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563660f-5b3b-40b1-b4a0-ccbd01cef0e4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:10.954821 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563660f-5b3b-40b1-b4a0-ccbd01cef0e4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:10.965024 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb8659-9c19-498b-92d2-1c78c109ee6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:10.966205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb8659-9c19-498b-92d2-1c78c109ee6f service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:10.966788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb8659-9c19-498b-92d2-1c78c109ee6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:10.967236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb8659-9c19-498b-92d2-1c78c109ee6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:10.980014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 815/1642] 10.222.0.17 () {58 vars in 1058 bytes} [Tue Jun 7 08:38:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:10.995288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea84c2-8a3d-491e-b033-a08b6e28f08c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:11.016231 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea84c2-8a3d-491e-b033-a08b6e28f08c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:11.038923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eca4aa-bb3f-4ab8-9e1a-a7430e16af4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:11.038923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eca4aa-bb3f-4ab8-9e1a-a7430e16af4a service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:11.038923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eca4aa-bb3f-4ab8-9e1a-a7430e16af4a service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:11.038923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eca4aa-bb3f-4ab8-9e1a-a7430e16af4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:11.067901 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eca4aa-bb3f-4ab8-9e1a-a7430e16af4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:11.072189 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eca4aa-bb3f-4ab8-9e1a-a7430e16af4a service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:11.096700 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb8659-9c19-498b-92d2-1c78c109ee6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:11.098784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb8659-9c19-498b-92d2-1c78c109ee6f service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:11.118150 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 816/1643] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:38:11.143878 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 828/1644] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:10 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:11.612914 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2351594-41e5-4d50-a34f-b1b9dc14759e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:11.612914 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2351594-41e5-4d50-a34f-b1b9dc14759e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:11.612914 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2351594-41e5-4d50-a34f-b1b9dc14759e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:11.612914 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d2351594-41e5-4d50-a34f-b1b9dc14759e None None] MFA Rules not processed for user `6c240d9d8a9a4d1a8bb43c1151edbe38`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:11.612914 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d2351594-41e5-4d50-a34f-b1b9dc14759e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:11.612914 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:11.618713 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 817/1645] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:11.620028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb8659-9c19-498b-92d2-1c78c109ee6f service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:11.666403 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22df60-c38e-4d41-bbeb-b6541db94d56 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:11.666898 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22df60-c38e-4d41-bbeb-b6541db94d56 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:11.667373 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22df60-c38e-4d41-bbeb-b6541db94d56 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:11.670340 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 818/1646] 10.222.0.17 () {54 vars in 912 bytes} [Tue Jun 7 08:38:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:38:11.677971 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc1881-7100-4da5-9ece-99ca4418408e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:11.678443 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc1881-7100-4da5-9ece-99ca4418408e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:11.679633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc1881-7100-4da5-9ece-99ca4418408e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:11.721280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-82dc1881-7100-4da5-9ece-99ca4418408e None None] MFA Rules not processed for user `ee42568b059b49a198ed2c01d5efdafd`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:11.728574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-82dc1881-7100-4da5-9ece-99ca4418408e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:11.763011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 819/1647] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jun 7 08:38:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:38:11.788159 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc1881-7100-4da5-9ece-99ca4418408e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:11.789776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-82dc1881-7100-4da5-9ece-99ca4418408e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:11.820425 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb8659-9c19-498b-92d2-1c78c109ee6f service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:11.826933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7204e-2c18-4636-9cad-dae2a90f5200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:11.829036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7204e-2c18-4636-9cad-dae2a90f5200 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:11.829592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7204e-2c18-4636-9cad-dae2a90f5200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:11.830095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7204e-2c18-4636-9cad-dae2a90f5200 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:11.831003 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7204e-2c18-4636-9cad-dae2a90f5200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:11.833576 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7204e-2c18-4636-9cad-dae2a90f5200 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:11.928349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc1881-7100-4da5-9ece-99ca4418408e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:11.949336 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68269cbe-90a7-4ed2-b0db-03bfc0ca17af service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4d9b5e3826049f2800f1b9424977d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:11.951396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68269cbe-90a7-4ed2-b0db-03bfc0ca17af service glance] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:11.955362 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68269cbe-90a7-4ed2-b0db-03bfc0ca17af service glance] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:11.955964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68269cbe-90a7-4ed2-b0db-03bfc0ca17af service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:11.980245 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 829/1648] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:11 2022] POST /identity/v3/projects => generated 405 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:11.983280 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7204e-2c18-4636-9cad-dae2a90f5200 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:11.993299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68269cbe-90a7-4ed2-b0db-03bfc0ca17af service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:11.995029 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68269cbe-90a7-4ed2-b0db-03bfc0ca17af service glance] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:12.039292 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 820/1649] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:12.129798 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7204e-2c18-4636-9cad-dae2a90f5200 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:12.142013 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca9661-3a65-4eab-a5b7-8ad405253f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:12.144153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca9661-3a65-4eab-a5b7-8ad405253f73 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:12.144933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca9661-3a65-4eab-a5b7-8ad405253f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:12.146989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca9661-3a65-4eab-a5b7-8ad405253f73 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:12.149687 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca9661-3a65-4eab-a5b7-8ad405253f73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:12.153355 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca9661-3a65-4eab-a5b7-8ad405253f73 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:12.365368 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 830/1650] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:11 2022] POST /identity/v3/users => generated 309 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:38:12.366574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68269cbe-90a7-4ed2-b0db-03bfc0ca17af service glance] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:12.400132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68269cbe-90a7-4ed2-b0db-03bfc0ca17af service glance] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:12.414682 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b09e7d-761c-4c09-8698-916c5346de79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:12.416642 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b09e7d-761c-4c09-8698-916c5346de79 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:12.416940 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b09e7d-761c-4c09-8698-916c5346de79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:12.417251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b09e7d-761c-4c09-8698-916c5346de79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:12.418067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b09e7d-761c-4c09-8698-916c5346de79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:12.425215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b09e7d-761c-4c09-8698-916c5346de79 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:12.443439 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 821/1651] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:12.448182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca9661-3a65-4eab-a5b7-8ad405253f73 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:12.456226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:12.462877 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca9661-3a65-4eab-a5b7-8ad405253f73 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:12.476674 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513b6d6-f41e-4c97-bec6-32b1709eab30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:12.477999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513b6d6-f41e-4c97-bec6-32b1709eab30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:12.478461 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513b6d6-f41e-4c97-bec6-32b1709eab30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:12.478886 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513b6d6-f41e-4c97-bec6-32b1709eab30 admin admin] PATH_INFO: `/v3/projects/5f754d50eb004f9e8d52207944098eb6/users/d3d9710e89134d5a94719da7503a2d33/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:12.508424 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513b6d6-f41e-4c97-bec6-32b1709eab30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f754d50eb004f9e8d52207944098eb6, user_id=d3d9710e89134d5a94719da7503a2d33, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:12.538938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513b6d6-f41e-4c97-bec6-32b1709eab30 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:12.633606 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 831/1652] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:12 2022] PUT /identity/v3/projects/5f754d50eb004f9e8d52207944098eb6/users/d3d9710e89134d5a94719da7503a2d33/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:12.637439 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b09e7d-761c-4c09-8698-916c5346de79 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:12.648785 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b09e7d-761c-4c09-8698-916c5346de79 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:12.660626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c0fd1-1c32-4d43-9b65-bb5e15a4b4aa None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:12.661354 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c0fd1-1c32-4d43-9b65-bb5e15a4b4aa None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:12.671474 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c0fd1-1c32-4d43-9b65-bb5e15a4b4aa None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:12.671474 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 832/1653] 10.222.0.17 () {54 vars in 912 bytes} [Tue Jun 7 08:38:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:38:12.671834 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ab941-3d72-44e2-a543-e3509a9524b6 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:12.672189 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ab941-3d72-44e2-a543-e3509a9524b6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:12.672500 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ab941-3d72-44e2-a543-e3509a9524b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:12.720896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a63ab941-3d72-44e2-a543-e3509a9524b6 None None] MFA Rules not processed for user `4a87f8bb06ee4a3ba7017422f4f1943e`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:12.722175 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a63ab941-3d72-44e2-a543-e3509a9524b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:12.736796 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7575e-1b60-4293-8573-ca72595feb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:12.737760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7575e-1b60-4293-8573-ca72595feb85 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:12.738183 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7575e-1b60-4293-8573-ca72595feb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:12.738789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7575e-1b60-4293-8573-ca72595feb85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:12.739922 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7575e-1b60-4293-8573-ca72595feb85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:12.742529 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7575e-1b60-4293-8573-ca72595feb85 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:12.755869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 822/1654] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:12.760959 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7575e-1b60-4293-8573-ca72595feb85 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:12.774544 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7575e-1b60-4293-8573-ca72595feb85 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:12.795288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9843d-d693-4b2a-920c-35afbafdf2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:12.796040 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 833/1655] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jun 7 08:38:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:38:12.796750 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9843d-d693-4b2a-920c-35afbafdf2ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:12.797143 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9843d-d693-4b2a-920c-35afbafdf2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:12.797912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9843d-d693-4b2a-920c-35afbafdf2ba admin admin] PATH_INFO: `/v3/projects/5f754d50eb004f9e8d52207944098eb6/users/d3d9710e89134d5a94719da7503a2d33/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:12.810047 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9843d-d693-4b2a-920c-35afbafdf2ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f754d50eb004f9e8d52207944098eb6, user_id=d3d9710e89134d5a94719da7503a2d33, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:12.812858 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9843d-d693-4b2a-920c-35afbafdf2ba admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:12.834596 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 823/1656] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:12 2022] PUT /identity/v3/projects/5f754d50eb004f9e8d52207944098eb6/users/d3d9710e89134d5a94719da7503a2d33/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:38:12.840306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ab941-3d72-44e2-a543-e3509a9524b6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:12.863317 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ab941-3d72-44e2-a543-e3509a9524b6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:12.933757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb9416-7aa7-44f7-a21a-2ea010540681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:12.942196 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb9416-7aa7-44f7-a21a-2ea010540681 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:12.942196 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb9416-7aa7-44f7-a21a-2ea010540681 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:12.942196 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb9416-7aa7-44f7-a21a-2ea010540681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:12.953175 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb9416-7aa7-44f7-a21a-2ea010540681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:12.955941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb9416-7aa7-44f7-a21a-2ea010540681 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:12.976391 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 834/1657] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:13.340679 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9843d-d693-4b2a-920c-35afbafdf2ba admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:13.559713 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:13.596874 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9843d-d693-4b2a-920c-35afbafdf2ba admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:13.620132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabd652-3952-40c4-9687-504b224a02f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:13.621483 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabd652-3952-40c4-9687-504b224a02f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:13.622047 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabd652-3952-40c4-9687-504b224a02f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:13.622583 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabd652-3952-40c4-9687-504b224a02f4 admin admin] PATH_INFO: `/v3/users/8b235adbf9ad4dbda55ea361e5c7fd2b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:13.624245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabd652-3952-40c4-9687-504b224a02f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b235adbf9ad4dbda55ea361e5c7fd2b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:13.625675 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-baabd652-3952-40c4-9687-504b224a02f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:38:13.625675 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 07 08:38:13.625675 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: Jun 07 08:38:13.627402 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabd652-3952-40c4-9687-504b224a02f4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:13.907253 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-baabd652-3952-40c4-9687-504b224a02f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b235adbf9ad4dbda55ea361e5c7fd2b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:13.910525 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-baabd652-3952-40c4-9687-504b224a02f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b235adbf9ad4dbda55ea361e5c7fd2b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:13.950817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-baabd652-3952-40c4-9687-504b224a02f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b235adbf9ad4dbda55ea361e5c7fd2b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:13.950817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-baabd652-3952-40c4-9687-504b224a02f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b235adbf9ad4dbda55ea361e5c7fd2b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:13.973445 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-baabd652-3952-40c4-9687-504b224a02f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b235adbf9ad4dbda55ea361e5c7fd2b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:14.074866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 824/1658] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:13 2022] DELETE /identity/v3/users/8b235adbf9ad4dbda55ea361e5c7fd2b => generated 0 bytes in 736 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:14.079669 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c6fba-58c1-46bd-88a5-d9c48120e326 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:14.080190 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c6fba-58c1-46bd-88a5-d9c48120e326 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:14.080190 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c6fba-58c1-46bd-88a5-d9c48120e326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:14.114290 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9e3c6fba-58c1-46bd-88a5-d9c48120e326 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:14.114290 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9e3c6fba-58c1-46bd-88a5-d9c48120e326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:14.151300 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:14.254158 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 835/1659] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:14.324731 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabd652-3952-40c4-9687-504b224a02f4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:14.326600 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-baabd652-3952-40c4-9687-504b224a02f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:14.476897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabd652-3952-40c4-9687-504b224a02f4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:14.488978 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ab8d4-f5c2-452e-bfa6-d46dd46d4617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:14.489975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ab8d4-f5c2-452e-bfa6-d46dd46d4617 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:14.491678 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ab8d4-f5c2-452e-bfa6-d46dd46d4617 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:14.492889 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ab8d4-f5c2-452e-bfa6-d46dd46d4617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:14.526708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ab8d4-f5c2-452e-bfa6-d46dd46d4617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:14.528824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ab8d4-f5c2-452e-bfa6-d46dd46d4617 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:14.543069 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 825/1660] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:14.551497 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d31a5-5260-42ac-aedb-c907733bd949 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:14.551889 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d31a5-5260-42ac-aedb-c907733bd949 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:14.552258 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d31a5-5260-42ac-aedb-c907733bd949 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:14.554693 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 836/1661] 10.222.0.17 () {54 vars in 960 bytes} [Tue Jun 7 08:38:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:38:14.562767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fac2c-f1cd-4eb8-953e-7e598903035e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:14.563198 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fac2c-f1cd-4eb8-953e-7e598903035e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:14.563751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fac2c-f1cd-4eb8-953e-7e598903035e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:14.596404 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-421fac2c-f1cd-4eb8-953e-7e598903035e None None] MFA Rules not processed for user `1fc0d7bd7e4547f4a8d90ffc2c2e1d03`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:14.611422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-421fac2c-f1cd-4eb8-953e-7e598903035e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:14.783014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 826/1662] 10.222.0.17 () {58 vars in 1057 bytes} [Tue Jun 7 08:38:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:14.784255 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d31a5-5260-42ac-aedb-c907733bd949 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:14.806087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d31a5-5260-42ac-aedb-c907733bd949 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:14.816530 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805249e-2727-4103-b568-8c68855a0e0b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc0d7bd7e4547f4a8d90ffc2c2e1d03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:14.817668 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805249e-2727-4103-b568-8c68855a0e0b service placement] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:14.834651 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805249e-2727-4103-b568-8c68855a0e0b service placement] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:14.835765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805249e-2727-4103-b568-8c68855a0e0b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:14.839297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2805249e-2727-4103-b568-8c68855a0e0b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:14.862161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805249e-2727-4103-b568-8c68855a0e0b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:14.865156 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805249e-2727-4103-b568-8c68855a0e0b service placement] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:14.878160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 837/1663] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:16.139639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fac2c-f1cd-4eb8-953e-7e598903035e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:16.142256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-421fac2c-f1cd-4eb8-953e-7e598903035e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:16.163843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:16.171861 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fac2c-f1cd-4eb8-953e-7e598903035e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:16.180004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb12841-2c4f-41ba-a44a-71800c2b682b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:16.189383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb12841-2c4f-41ba-a44a-71800c2b682b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:16.189885 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb12841-2c4f-41ba-a44a-71800c2b682b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:16.190885 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb12841-2c4f-41ba-a44a-71800c2b682b admin admin] PATH_INFO: `/v3/projects/352bab09470a4f67bec618675f8d5fa0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:16.197480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb12841-2c4f-41ba-a44a-71800c2b682b admin admin] RBAC: Authorizing `identity:delete_project(project_id=352bab09470a4f67bec618675f8d5fa0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:16.200675 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb12841-2c4f-41ba-a44a-71800c2b682b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:16.467493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3fb12841-2c4f-41ba-a44a-71800c2b682b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '352bab09470a4f67bec618675f8d5fa0', 'request_id': 'req-3fb12841-2c4f-41ba-a44a-71800c2b682b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:16.492802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3fb12841-2c4f-41ba-a44a-71800c2b682b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '352bab09470a4f67bec618675f8d5fa0', 'request_id': 'req-3fb12841-2c4f-41ba-a44a-71800c2b682b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:16.531192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3fb12841-2c4f-41ba-a44a-71800c2b682b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '352bab09470a4f67bec618675f8d5fa0', 'request_id': 'req-3fb12841-2c4f-41ba-a44a-71800c2b682b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:16.677436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bc054-bd26-4f6f-8609-3c390302ca7c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:16.679523 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bc054-bd26-4f6f-8609-3c390302ca7c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:16.680056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bc054-bd26-4f6f-8609-3c390302ca7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:16.683005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3fb12841-2c4f-41ba-a44a-71800c2b682b admin admin] The token cache is being invalidate because project 352bab09470a4f67bec618675f8d5fa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:16.688779 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3fb12841-2c4f-41ba-a44a-71800c2b682b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:16.688779 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3fb12841-2c4f-41ba-a44a-71800c2b682b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:16.688779 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 827/1664] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:16 2022] DELETE /identity/v3/projects/352bab09470a4f67bec618675f8d5fa0 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:16.692555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb12841-2c4f-41ba-a44a-71800c2b682b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:16.700700 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3fb12841-2c4f-41ba-a44a-71800c2b682b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:16.734296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-319bc054-bd26-4f6f-8609-3c390302ca7c None None] MFA Rules not processed for user `d3d9710e89134d5a94719da7503a2d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:16.735782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-319bc054-bd26-4f6f-8609-3c390302ca7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:16.926974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb12841-2c4f-41ba-a44a-71800c2b682b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:16.946158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18a427-03db-4b7a-8a46-976617ca81ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:16.949135 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18a427-03db-4b7a-8a46-976617ca81ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:16.949432 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18a427-03db-4b7a-8a46-976617ca81ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:16.949715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18a427-03db-4b7a-8a46-976617ca81ae admin admin] PATH_INFO: `/v3/users/1723dcdb1aef47bba89fdcc213f091b5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:16.951008 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 838/1665] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:16 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:38:16.959798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18a427-03db-4b7a-8a46-976617ca81ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=1723dcdb1aef47bba89fdcc213f091b5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:16.965265 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18a427-03db-4b7a-8a46-976617ca81ae admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:17.205802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eb18a427-03db-4b7a-8a46-976617ca81ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1723dcdb1aef47bba89fdcc213f091b5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:17.208096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eb18a427-03db-4b7a-8a46-976617ca81ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1723dcdb1aef47bba89fdcc213f091b5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:17.236676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eb18a427-03db-4b7a-8a46-976617ca81ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1723dcdb1aef47bba89fdcc213f091b5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:17.237943 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eb18a427-03db-4b7a-8a46-976617ca81ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1723dcdb1aef47bba89fdcc213f091b5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:17.271654 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eb18a427-03db-4b7a-8a46-976617ca81ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1723dcdb1aef47bba89fdcc213f091b5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:17.380298 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 828/1666] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:16 2022] DELETE /identity/v3/users/1723dcdb1aef47bba89fdcc213f091b5 => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:18.013744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a7a05-90fc-41b5-add8-9265e077fbf2 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:18.014331 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a7a05-90fc-41b5-add8-9265e077fbf2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:18.015338 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a7a05-90fc-41b5-add8-9265e077fbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:18.095032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-0e3a7a05-90fc-41b5-add8-9265e077fbf2 None None] MFA Rules not processed for user `055ec8a54bdf400fac0a46b0e5497f1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:18.096382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0e3a7a05-90fc-41b5-add8-9265e077fbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:18.251326 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 839/1667] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:18.273155 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5295ae-49d7-44eb-8f65-7650e48eed23 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:18.273155 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5295ae-49d7-44eb-8f65-7650e48eed23 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:18.273155 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5295ae-49d7-44eb-8f65-7650e48eed23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:18.315393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:18.333278 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0f5295ae-49d7-44eb-8f65-7650e48eed23 None None] MFA Rules not processed for user `055ec8a54bdf400fac0a46b0e5497f1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:18.335914 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0f5295ae-49d7-44eb-8f65-7650e48eed23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:18.386942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 829/1668] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:18.714091 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a7a05-90fc-41b5-add8-9265e077fbf2 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:18.714091 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0e3a7a05-90fc-41b5-add8-9265e077fbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:18.730624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5295ae-49d7-44eb-8f65-7650e48eed23 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:18.732842 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0f5295ae-49d7-44eb-8f65-7650e48eed23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:18.774086 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:18.857684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a7a05-90fc-41b5-add8-9265e077fbf2 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:18.862983 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05fc32-72ed-42eb-95a7-c27777b517e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:18.864168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05fc32-72ed-42eb-95a7-c27777b517e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:18.864597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05fc32-72ed-42eb-95a7-c27777b517e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:18.865040 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05fc32-72ed-42eb-95a7-c27777b517e5 admin admin] PATH_INFO: `/v3/projects/8f614aff98a742f2b2d9305804396efb` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:18.867376 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05fc32-72ed-42eb-95a7-c27777b517e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f614aff98a742f2b2d9305804396efb)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:18.868701 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: WARNING py.warnings [None req-4c05fc32-72ed-42eb-95a7-c27777b517e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 08:38:18.868701 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: warnings.warn(msg) Jun 07 08:38:18.868701 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: Jun 07 08:38:18.870517 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05fc32-72ed-42eb-95a7-c27777b517e5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:18.907909 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5295ae-49d7-44eb-8f65-7650e48eed23 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:18.928080 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24344f0-91d9-48f6-bbfc-3808ed7f6055 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:18.928845 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24344f0-91d9-48f6-bbfc-3808ed7f6055 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:18.929669 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24344f0-91d9-48f6-bbfc-3808ed7f6055 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:18.929948 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24344f0-91d9-48f6-bbfc-3808ed7f6055 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:18.944563 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24344f0-91d9-48f6-bbfc-3808ed7f6055 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:18.946767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24344f0-91d9-48f6-bbfc-3808ed7f6055 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:18.960075 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 830/1669] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:38:19.087765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4c05fc32-72ed-42eb-95a7-c27777b517e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f614aff98a742f2b2d9305804396efb', 'request_id': 'req-4c05fc32-72ed-42eb-95a7-c27777b517e5'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:19.141765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4c05fc32-72ed-42eb-95a7-c27777b517e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f614aff98a742f2b2d9305804396efb', 'request_id': 'req-4c05fc32-72ed-42eb-95a7-c27777b517e5'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:19.152388 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70667420-1a58-497a-af31-dcdb9d781383 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:19.153235 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70667420-1a58-497a-af31-dcdb9d781383 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:19.153729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70667420-1a58-497a-af31-dcdb9d781383 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:19.156364 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 831/1670] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jun 7 08:38:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:38:19.201281 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af788f-082c-4179-bf92-b77f61ce97d5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:19.201574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af788f-082c-4179-bf92-b77f61ce97d5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:19.201824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af788f-082c-4179-bf92-b77f61ce97d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:19.238359 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4c05fc32-72ed-42eb-95a7-c27777b517e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f614aff98a742f2b2d9305804396efb', 'request_id': 'req-4c05fc32-72ed-42eb-95a7-c27777b517e5'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:19.257529 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c5af788f-082c-4179-bf92-b77f61ce97d5 None None] MFA Rules not processed for user `1fc0d7bd7e4547f4a8d90ffc2c2e1d03`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:19.258560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c5af788f-082c-4179-bf92-b77f61ce97d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:19.294261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4c05fc32-72ed-42eb-95a7-c27777b517e5 admin admin] The token cache is being invalidate because project 8f614aff98a742f2b2d9305804396efb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:19.296073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4c05fc32-72ed-42eb-95a7-c27777b517e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:19.308756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4c05fc32-72ed-42eb-95a7-c27777b517e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:19.308756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 840/1671] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:18 2022] DELETE /identity/v3/projects/8f614aff98a742f2b2d9305804396efb => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:19.312446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a65a1-b741-4cef-b494-eefbd5bbcc2c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:19.315698 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a65a1-b741-4cef-b494-eefbd5bbcc2c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:19.316025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a65a1-b741-4cef-b494-eefbd5bbcc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:19.390145 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 832/1672] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jun 7 08:38:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:19.392672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-620a65a1-b741-4cef-b494-eefbd5bbcc2c None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:19.396582 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-620a65a1-b741-4cef-b494-eefbd5bbcc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:19.423061 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af788f-082c-4179-bf92-b77f61ce97d5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:19.425363 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c5af788f-082c-4179-bf92-b77f61ce97d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:19.449057 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af788f-082c-4179-bf92-b77f61ce97d5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:19.462383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194495b-46d4-4477-8c5f-e7a6c4ac1a52 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc0d7bd7e4547f4a8d90ffc2c2e1d03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:19.463432 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194495b-46d4-4477-8c5f-e7a6c4ac1a52 service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:19.463852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194495b-46d4-4477-8c5f-e7a6c4ac1a52 service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:19.464264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194495b-46d4-4477-8c5f-e7a6c4ac1a52 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:19.477249 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194495b-46d4-4477-8c5f-e7a6c4ac1a52 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:19.479700 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194495b-46d4-4477-8c5f-e7a6c4ac1a52 service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:19.507170 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 833/1673] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:19.612353 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 841/1674] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:19.614400 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4a127-4213-4d35-b152-4b26bdff0f61 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:19.614655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4a127-4213-4d35-b152-4b26bdff0f61 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:19.615202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4a127-4213-4d35-b152-4b26bdff0f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:19.692233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6af4a127-4213-4d35-b152-4b26bdff0f61 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:19.693628 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6af4a127-4213-4d35-b152-4b26bdff0f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:19.713348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 834/1675] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:19.716968 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a65a1-b741-4cef-b494-eefbd5bbcc2c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:19.731532 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a65a1-b741-4cef-b494-eefbd5bbcc2c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:19.744978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fd1b8-c1db-4fbd-987c-d5b39056f262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:19.745757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fd1b8-c1db-4fbd-987c-d5b39056f262 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:19.746044 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fd1b8-c1db-4fbd-987c-d5b39056f262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:19.746747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fd1b8-c1db-4fbd-987c-d5b39056f262 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:19.747522 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031fd1b8-c1db-4fbd-987c-d5b39056f262 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:19.752135 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031fd1b8-c1db-4fbd-987c-d5b39056f262 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:19.753866 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031fd1b8-c1db-4fbd-987c-d5b39056f262 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:19.788026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 842/1676] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:19.788966 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4a127-4213-4d35-b152-4b26bdff0f61 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:19.801938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4a127-4213-4d35-b152-4b26bdff0f61 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:19.815127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d35b38-012e-42dc-b9fe-2b116a639cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:19.816961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d35b38-012e-42dc-b9fe-2b116a639cfd admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:19.822009 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d35b38-012e-42dc-b9fe-2b116a639cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:19.822009 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d35b38-012e-42dc-b9fe-2b116a639cfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:19.822009 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d35b38-012e-42dc-b9fe-2b116a639cfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:19.826775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d35b38-012e-42dc-b9fe-2b116a639cfd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:19.957252 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 835/1677] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:19 2022] POST /identity/v3/projects => generated 363 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:19.963269 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fd1b8-c1db-4fbd-987c-d5b39056f262 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:19.978300 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fd1b8-c1db-4fbd-987c-d5b39056f262 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:20.064718 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:20.116180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117216b2-0310-4d3a-aade-1c3554190b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:20.116889 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117216b2-0310-4d3a-aade-1c3554190b7b admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:20.117025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117216b2-0310-4d3a-aade-1c3554190b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:20.117280 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117216b2-0310-4d3a-aade-1c3554190b7b admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:20.118007 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117216b2-0310-4d3a-aade-1c3554190b7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:20.123914 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117216b2-0310-4d3a-aade-1c3554190b7b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:20.216132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443691be-8dfc-43e7-a39e-815144ee3232 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:20.216423 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443691be-8dfc-43e7-a39e-815144ee3232 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:20.216702 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443691be-8dfc-43e7-a39e-815144ee3232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:20.251971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 843/1678] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:19 2022] POST /identity/v3/users => generated 282 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:20.256144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117216b2-0310-4d3a-aade-1c3554190b7b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:20.278270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-443691be-8dfc-43e7-a39e-815144ee3232 None None] MFA Rules not processed for user `bbf0ce396885472bb5d225aa009043f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:20.279640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-443691be-8dfc-43e7-a39e-815144ee3232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:20.287833 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117216b2-0310-4d3a-aade-1c3554190b7b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:20.314859 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3ca71-d92a-466e-b7c6-7135b061f1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:20.321340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3ca71-d92a-466e-b7c6-7135b061f1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:20.321628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3ca71-d92a-466e-b7c6-7135b061f1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:20.321903 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3ca71-d92a-466e-b7c6-7135b061f1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:20.322753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3ca71-d92a-466e-b7c6-7135b061f1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:20.325814 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3ca71-d92a-466e-b7c6-7135b061f1b0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:20.363780 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 844/1679] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:20.364854 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3ca71-d92a-466e-b7c6-7135b061f1b0 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:20.396915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:20.399370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3ca71-d92a-466e-b7c6-7135b061f1b0 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:20.421195 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfaba48-86f0-40c6-a64c-733949915c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:20.424761 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaba48-86f0-40c6-a64c-733949915c28 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:20.425096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaba48-86f0-40c6-a64c-733949915c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:20.426840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaba48-86f0-40c6-a64c-733949915c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:20.430932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfaba48-86f0-40c6-a64c-733949915c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:20.432505 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 836/1680] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jun 7 08:38:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:20.448293 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfaba48-86f0-40c6-a64c-733949915c28 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:20.486415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443691be-8dfc-43e7-a39e-815144ee3232 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:20.487557 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 845/1681] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:20.499388 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443691be-8dfc-43e7-a39e-815144ee3232 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:20.504617 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09722532-3a3d-45d5-adde-b0484f99ac6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:20.505585 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09722532-3a3d-45d5-adde-b0484f99ac6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:20.505869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09722532-3a3d-45d5-adde-b0484f99ac6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:20.506146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09722532-3a3d-45d5-adde-b0484f99ac6c admin admin] PATH_INFO: `/v3/projects/7d4334cc21564921b3d007b9dfbb9260/users/2cd827b8e9f34dae8bdafdf3f8f9ba07/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:20.530937 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09722532-3a3d-45d5-adde-b0484f99ac6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d4334cc21564921b3d007b9dfbb9260, user_id=2cd827b8e9f34dae8bdafdf3f8f9ba07, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:20.534829 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09722532-3a3d-45d5-adde-b0484f99ac6c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:20.561652 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f29b7c-b55a-42bc-a84c-c9cd4484cd5f None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:20.562104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f29b7c-b55a-42bc-a84c-c9cd4484cd5f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:20.562995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f29b7c-b55a-42bc-a84c-c9cd4484cd5f None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:20.564638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 846/1682] 10.222.0.17 () {54 vars in 912 bytes} [Tue Jun 7 08:38:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:38:20.572000 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d1adf-453c-4507-a4cd-92174230a586 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:20.572506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d1adf-453c-4507-a4cd-92174230a586 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:20.572978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d1adf-453c-4507-a4cd-92174230a586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:20.603080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-6d5d1adf-453c-4507-a4cd-92174230a586 None None] MFA Rules not processed for user `4a87f8bb06ee4a3ba7017422f4f1943e`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:20.606758 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6d5d1adf-453c-4507-a4cd-92174230a586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:20.653448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 837/1683] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:20 2022] PUT /identity/v3/projects/7d4334cc21564921b3d007b9dfbb9260/users/2cd827b8e9f34dae8bdafdf3f8f9ba07/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:20.666488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b6a27-9095-432f-ac01-ea5c5d2230bc None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:20.666919 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b6a27-9095-432f-ac01-ea5c5d2230bc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:20.667357 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b6a27-9095-432f-ac01-ea5c5d2230bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:20.758226 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-979b6a27-9095-432f-ac01-ea5c5d2230bc None None] MFA Rules not processed for user `2cd827b8e9f34dae8bdafdf3f8f9ba07`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:20.759890 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 847/1684] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jun 7 08:38:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:38:20.761149 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-979b6a27-9095-432f-ac01-ea5c5d2230bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:20.866224 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d1adf-453c-4507-a4cd-92174230a586 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:20.869322 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6d5d1adf-453c-4507-a4cd-92174230a586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:20.885987 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d1adf-453c-4507-a4cd-92174230a586 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:20.926668 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 838/1685] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:20 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:38:20.931219 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b6a27-9095-432f-ac01-ea5c5d2230bc None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:20.989087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fa929-ce34-4aa9-a3b6-a5bd09b224d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:20.990082 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fa929-ce34-4aa9-a3b6-a5bd09b224d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:20.990574 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fa929-ce34-4aa9-a3b6-a5bd09b224d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:20.991030 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fa929-ce34-4aa9-a3b6-a5bd09b224d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:21.020375 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fa929-ce34-4aa9-a3b6-a5bd09b224d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:21.024061 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fa929-ce34-4aa9-a3b6-a5bd09b224d0 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:21.050966 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 848/1686] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:21.052132 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fa929-ce34-4aa9-a3b6-a5bd09b224d0 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:21.056652 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-470fa929-ce34-4aa9-a3b6-a5bd09b224d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:21.093228 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b6a27-9095-432f-ac01-ea5c5d2230bc None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:21.106627 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03f9be-ddf7-44ba-b3c8-f66e0a163e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:21.107479 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03f9be-ddf7-44ba-b3c8-f66e0a163e65 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:21.107772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03f9be-ddf7-44ba-b3c8-f66e0a163e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:21.108048 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03f9be-ddf7-44ba-b3c8-f66e0a163e65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:21.108856 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03f9be-ddf7-44ba-b3c8-f66e0a163e65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:21.115190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03f9be-ddf7-44ba-b3c8-f66e0a163e65 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:21.192684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fa929-ce34-4aa9-a3b6-a5bd09b224d0 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:21.202482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326584bb-3427-48c3-bdcc-12368b4e9170 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334dc47ae2e643ccb44d25c6b5906f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:21.203162 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326584bb-3427-48c3-bdcc-12368b4e9170 service swift] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:21.203549 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326584bb-3427-48c3-bdcc-12368b4e9170 service swift] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:21.203897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326584bb-3427-48c3-bdcc-12368b4e9170 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:21.261791 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 839/1687] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:20 2022] POST /identity/v3/projects => generated 363 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:21.265691 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03f9be-ddf7-44ba-b3c8-f66e0a163e65 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:21.339754 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326584bb-3427-48c3-bdcc-12368b4e9170 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:21.352319 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326584bb-3427-48c3-bdcc-12368b4e9170 service swift] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:21.376639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03f9be-ddf7-44ba-b3c8-f66e0a163e65 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:21.378256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 849/1688] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jun 7 08:38:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:38:21.383559 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8ad3d-3358-4fe3-b0e3-08a8af6d871c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:21.384366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8ad3d-3358-4fe3-b0e3-08a8af6d871c admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:21.384787 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8ad3d-3358-4fe3-b0e3-08a8af6d871c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:21.384981 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8ad3d-3358-4fe3-b0e3-08a8af6d871c admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:21.385815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8ad3d-3358-4fe3-b0e3-08a8af6d871c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:21.388348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8ad3d-3358-4fe3-b0e3-08a8af6d871c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:21.527564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 840/1689] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:21 2022] POST /identity/v3/users => generated 288 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:21.531395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326584bb-3427-48c3-bdcc-12368b4e9170 service swift] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:21.546707 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326584bb-3427-48c3-bdcc-12368b4e9170 service swift] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:21.562756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5bfa7-33f0-451e-b93b-2807df828dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:21.563356 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5bfa7-33f0-451e-b93b-2807df828dfe admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:21.563645 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5bfa7-33f0-451e-b93b-2807df828dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:21.565582 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5bfa7-33f0-451e-b93b-2807df828dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:21.573348 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5bfa7-33f0-451e-b93b-2807df828dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:21.578177 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5bfa7-33f0-451e-b93b-2807df828dfe admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:21.643847 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 850/1690] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:21.644976 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8ad3d-3358-4fe3-b0e3-08a8af6d871c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:21.666714 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8ad3d-3358-4fe3-b0e3-08a8af6d871c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:21.735070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d496f28-bee1-4fde-8e2b-b1693bc4fe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:21.736049 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d496f28-bee1-4fde-8e2b-b1693bc4fe2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:21.736458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d496f28-bee1-4fde-8e2b-b1693bc4fe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:21.736611 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d496f28-bee1-4fde-8e2b-b1693bc4fe2d admin admin] PATH_INFO: `/v3/projects/2f84feda4dd243c5b335be949822e693/users/fc2b283c30fa47b990c14c6fad85412a/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:21.761785 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:21.799810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d496f28-bee1-4fde-8e2b-b1693bc4fe2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f84feda4dd243c5b335be949822e693, user_id=fc2b283c30fa47b990c14c6fad85412a, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:21.820623 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d496f28-bee1-4fde-8e2b-b1693bc4fe2d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:21.945999 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 841/1691] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:21 2022] PUT /identity/v3/projects/2f84feda4dd243c5b335be949822e693/users/fc2b283c30fa47b990c14c6fad85412a/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:21.955311 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5bfa7-33f0-451e-b93b-2807df828dfe admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:21.967665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5bfa7-33f0-451e-b93b-2807df828dfe admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:22.206172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d496f28-bee1-4fde-8e2b-b1693bc4fe2d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:22.208053 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5d496f28-bee1-4fde-8e2b-b1693bc4fe2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:22.217464 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:22.222184 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05745f98-0643-40fc-99d3-bc1c03f851fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:22.223088 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05745f98-0643-40fc-99d3-bc1c03f851fc admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:22.223381 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05745f98-0643-40fc-99d3-bc1c03f851fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:22.226279 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05745f98-0643-40fc-99d3-bc1c03f851fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:22.226279 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05745f98-0643-40fc-99d3-bc1c03f851fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:22.233094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05745f98-0643-40fc-99d3-bc1c03f851fc admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:22.244736 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d496f28-bee1-4fde-8e2b-b1693bc4fe2d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:22.287406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0fbca-c3e8-4e0e-bb85-b42ddc4d40e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:22.287406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0fbca-c3e8-4e0e-bb85-b42ddc4d40e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:22.287406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0fbca-c3e8-4e0e-bb85-b42ddc4d40e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:22.287406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0fbca-c3e8-4e0e-bb85-b42ddc4d40e3 admin admin] PATH_INFO: `/v3/users/89b9c8a15aee4129a24abd82eaa54c05` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:22.287819 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 851/1692] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:22.290910 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05745f98-0643-40fc-99d3-bc1c03f851fc admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:22.292223 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f0fbca-c3e8-4e0e-bb85-b42ddc4d40e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89b9c8a15aee4129a24abd82eaa54c05)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:22.299670 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f0fbca-c3e8-4e0e-bb85-b42ddc4d40e3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:22.301893 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-05745f98-0643-40fc-99d3-bc1c03f851fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:22.357546 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05745f98-0643-40fc-99d3-bc1c03f851fc admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:22.381909 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63453fe-323a-4d14-8b8c-b8124b70a9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:22.384472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63453fe-323a-4d14-8b8c-b8124b70a9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:22.388207 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63453fe-323a-4d14-8b8c-b8124b70a9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:22.389436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63453fe-323a-4d14-8b8c-b8124b70a9fd admin admin] PATH_INFO: `/v3/users/6c240d9d8a9a4d1a8bb43c1151edbe38` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:22.393042 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63453fe-323a-4d14-8b8c-b8124b70a9fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c240d9d8a9a4d1a8bb43c1151edbe38)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:22.396542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63453fe-323a-4d14-8b8c-b8124b70a9fd admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:22.677182 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-71f0fbca-c3e8-4e0e-bb85-b42ddc4d40e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89b9c8a15aee4129a24abd82eaa54c05'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:22.682006 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-71f0fbca-c3e8-4e0e-bb85-b42ddc4d40e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89b9c8a15aee4129a24abd82eaa54c05'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:22.702554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-71f0fbca-c3e8-4e0e-bb85-b42ddc4d40e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89b9c8a15aee4129a24abd82eaa54c05'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:22.705304 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-71f0fbca-c3e8-4e0e-bb85-b42ddc4d40e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89b9c8a15aee4129a24abd82eaa54c05'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:22.727186 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-71f0fbca-c3e8-4e0e-bb85-b42ddc4d40e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89b9c8a15aee4129a24abd82eaa54c05'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:22.789267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e63453fe-323a-4d14-8b8c-b8124b70a9fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c240d9d8a9a4d1a8bb43c1151edbe38'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:22.789267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e63453fe-323a-4d14-8b8c-b8124b70a9fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c240d9d8a9a4d1a8bb43c1151edbe38'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:22.811237 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e63453fe-323a-4d14-8b8c-b8124b70a9fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c240d9d8a9a4d1a8bb43c1151edbe38'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:22.813424 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e63453fe-323a-4d14-8b8c-b8124b70a9fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c240d9d8a9a4d1a8bb43c1151edbe38'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:22.852460 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 842/1693] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:22 2022] DELETE /identity/v3/users/89b9c8a15aee4129a24abd82eaa54c05 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:22.854827 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0fbca-c3e8-4e0e-bb85-b42ddc4d40e3 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:22.861394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-71f0fbca-c3e8-4e0e-bb85-b42ddc4d40e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:22.939604 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e63453fe-323a-4d14-8b8c-b8124b70a9fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c240d9d8a9a4d1a8bb43c1151edbe38'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:23.017952 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 852/1694] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:22 2022] DELETE /identity/v3/users/6c240d9d8a9a4d1a8bb43c1151edbe38 => generated 0 bytes in 725 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:23.019912 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71b2b9-6289-43b6-9ea1-834161961b3c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:23.020353 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71b2b9-6289-43b6-9ea1-834161961b3c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:23.021465 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71b2b9-6289-43b6-9ea1-834161961b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:23.082350 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-8e71b2b9-6289-43b6-9ea1-834161961b3c None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:23.084394 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8e71b2b9-6289-43b6-9ea1-834161961b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:23.350124 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:23.351973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0fbca-c3e8-4e0e-bb85-b42ddc4d40e3 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:23.393379 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:23.423091 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 853/1695] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:23.470368 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71b2b9-6289-43b6-9ea1-834161961b3c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:23.473561 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8e71b2b9-6289-43b6-9ea1-834161961b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:23.506497 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c6100-ce52-4a16-a255-b1b56ae0d226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:23.510597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c6100-ce52-4a16-a255-b1b56ae0d226 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:23.511672 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c6100-ce52-4a16-a255-b1b56ae0d226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:23.512115 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c6100-ce52-4a16-a255-b1b56ae0d226 admin admin] PATH_INFO: `/v3/projects/2f84feda4dd243c5b335be949822e693/users/fc2b283c30fa47b990c14c6fad85412a/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:23.521295 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c6100-ce52-4a16-a255-b1b56ae0d226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f84feda4dd243c5b335be949822e693, user_id=fc2b283c30fa47b990c14c6fad85412a, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:23.530446 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c6100-ce52-4a16-a255-b1b56ae0d226 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:23.565732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 843/1696] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:22 2022] PUT /identity/v3/projects/2f84feda4dd243c5b335be949822e693/users/fc2b283c30fa47b990c14c6fad85412a/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:23.585029 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485eb795-6eaa-4654-bf38-21285ab3dc01 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:23.585029 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485eb795-6eaa-4654-bf38-21285ab3dc01 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:23.585029 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485eb795-6eaa-4654-bf38-21285ab3dc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:23.674839 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-485eb795-6eaa-4654-bf38-21285ab3dc01 None None] MFA Rules not processed for user `fc2b283c30fa47b990c14c6fad85412a`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:23.677436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71b2b9-6289-43b6-9ea1-834161961b3c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:23.679543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-485eb795-6eaa-4654-bf38-21285ab3dc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:23.892647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd735a8f-34c2-42b1-a8dd-fd8e1a23baa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:23.894534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd735a8f-34c2-42b1-a8dd-fd8e1a23baa9 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:23.898502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd735a8f-34c2-42b1-a8dd-fd8e1a23baa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:23.899862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd735a8f-34c2-42b1-a8dd-fd8e1a23baa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:24.007653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 844/1697] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:23 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:24.009884 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc11fc-f81c-4af4-b288-c596a2118e2a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:24.009884 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc11fc-f81c-4af4-b288-c596a2118e2a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:24.009884 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc11fc-f81c-4af4-b288-c596a2118e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:24.086235 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d3dc11fc-f81c-4af4-b288-c596a2118e2a None None] MFA Rules not processed for user `0b2efb9857cf495f8b51a96b5f868d44`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:24.086235 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d3dc11fc-f81c-4af4-b288-c596a2118e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:24.125486 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd735a8f-34c2-42b1-a8dd-fd8e1a23baa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:24.130786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd735a8f-34c2-42b1-a8dd-fd8e1a23baa9 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:24.173516 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 854/1698] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 703 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:24.175263 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f72c82-5cfd-4244-8c06-456e4b88dfe4 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:24.184669 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f72c82-5cfd-4244-8c06-456e4b88dfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:24.189879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f72c82-5cfd-4244-8c06-456e4b88dfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:24.234409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c2f72c82-5cfd-4244-8c06-456e4b88dfe4 None None] MFA Rules not processed for user `4a87f8bb06ee4a3ba7017422f4f1943e`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:24.235886 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c2f72c82-5cfd-4244-8c06-456e4b88dfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:24.261703 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 855/1699] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Jun 7 08:38:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:24.263486 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f72c82-5cfd-4244-8c06-456e4b88dfe4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:24.265123 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c2f72c82-5cfd-4244-8c06-456e4b88dfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:24.306690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 845/1700] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:24.306690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc11fc-f81c-4af4-b288-c596a2118e2a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:24.311585 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d3dc11fc-f81c-4af4-b288-c596a2118e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:24.360975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc11fc-f81c-4af4-b288-c596a2118e2a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:24.372809 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fadc9-c0b7-4bad-b714-7e098a8f4089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:24.374250 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fadc9-c0b7-4bad-b714-7e098a8f4089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:24.374695 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fadc9-c0b7-4bad-b714-7e098a8f4089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:24.375151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fadc9-c0b7-4bad-b714-7e098a8f4089 admin admin] PATH_INFO: `/v3/projects/448b9f27e4bc4870aebf03e2f8bca11e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:24.376982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23fadc9-c0b7-4bad-b714-7e098a8f4089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=448b9f27e4bc4870aebf03e2f8bca11e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:24.379128 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23fadc9-c0b7-4bad-b714-7e098a8f4089 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:24.487623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f72c82-5cfd-4244-8c06-456e4b88dfe4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:24.509677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc493ff-f8a9-425d-aefa-c91ae5cae793 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:24.510650 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc493ff-f8a9-425d-aefa-c91ae5cae793 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:24.511661 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc493ff-f8a9-425d-aefa-c91ae5cae793 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:24.512686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc493ff-f8a9-425d-aefa-c91ae5cae793 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:24.538099 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc493ff-f8a9-425d-aefa-c91ae5cae793 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:24.539964 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc493ff-f8a9-425d-aefa-c91ae5cae793 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:24.593262 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 856/1701] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:24 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:24.595945 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec04c8-d4cb-4f64-bf41-e4e05203b03a None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:24.596463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec04c8-d4cb-4f64-bf41-e4e05203b03a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:24.597683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec04c8-d4cb-4f64-bf41-e4e05203b03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:24.606283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d23fadc9-c0b7-4bad-b714-7e098a8f4089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '448b9f27e4bc4870aebf03e2f8bca11e', 'request_id': 'req-d23fadc9-c0b7-4bad-b714-7e098a8f4089'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:24.621328 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:24.627862 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d23fadc9-c0b7-4bad-b714-7e098a8f4089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '448b9f27e4bc4870aebf03e2f8bca11e', 'request_id': 'req-d23fadc9-c0b7-4bad-b714-7e098a8f4089'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:24.629987 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-61ec04c8-d4cb-4f64-bf41-e4e05203b03a None None] MFA Rules not processed for user `055ec8a54bdf400fac0a46b0e5497f1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:24.632690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-61ec04c8-d4cb-4f64-bf41-e4e05203b03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:24.651073 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d23fadc9-c0b7-4bad-b714-7e098a8f4089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '448b9f27e4bc4870aebf03e2f8bca11e', 'request_id': 'req-d23fadc9-c0b7-4bad-b714-7e098a8f4089'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:24.764440 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:24.772646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d23fadc9-c0b7-4bad-b714-7e098a8f4089 admin admin] The token cache is being invalidate because project 448b9f27e4bc4870aebf03e2f8bca11e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:24.773379 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d23fadc9-c0b7-4bad-b714-7e098a8f4089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:24.774720 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d23fadc9-c0b7-4bad-b714-7e098a8f4089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:24.778664 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 846/1702] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:24 2022] DELETE /identity/v3/projects/448b9f27e4bc4870aebf03e2f8bca11e => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:24.782088 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fadc9-c0b7-4bad-b714-7e098a8f4089 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:24.787674 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d23fadc9-c0b7-4bad-b714-7e098a8f4089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:24.876689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 857/1703] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:24.877882 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec04c8-d4cb-4f64-bf41-e4e05203b03a None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:24.886584 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-61ec04c8-d4cb-4f64-bf41-e4e05203b03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:25.062469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec04c8-d4cb-4f64-bf41-e4e05203b03a None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:25.082240 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a6d30-7e7c-4b29-a58b-d54798d61d48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:25.083224 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a6d30-7e7c-4b29-a58b-d54798d61d48 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:25.084853 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a6d30-7e7c-4b29-a58b-d54798d61d48 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:25.085604 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a6d30-7e7c-4b29-a58b-d54798d61d48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:25.091189 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d04a6d30-7e7c-4b29-a58b-d54798d61d48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:25.157176 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fadc9-c0b7-4bad-b714-7e098a8f4089 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:25.169729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1b040-d136-4631-becb-96119d7d4595 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:25.171873 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1b040-d136-4631-becb-96119d7d4595 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:25.172901 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1b040-d136-4631-becb-96119d7d4595 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:25.174240 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1b040-d136-4631-becb-96119d7d4595 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:25.187872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-37f1b040-d136-4631-becb-96119d7d4595 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:25.222362 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1b040-d136-4631-becb-96119d7d4595 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:25.222362 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1b040-d136-4631-becb-96119d7d4595 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:25.233603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 847/1704] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:25.234613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1b040-d136-4631-becb-96119d7d4595 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:25.236393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-37f1b040-d136-4631-becb-96119d7d4595 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:25.244550 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:25.383371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a6d30-7e7c-4b29-a58b-d54798d61d48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:25.383371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a6d30-7e7c-4b29-a58b-d54798d61d48 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:25.384747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 858/1705] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:25.387300 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a6d30-7e7c-4b29-a58b-d54798d61d48 service nova] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:25.391636 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d04a6d30-7e7c-4b29-a58b-d54798d61d48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:25.563173 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1b040-d136-4631-becb-96119d7d4595 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:25.582015 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242c00e-8765-4ddd-94ff-1162c0dc5ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:25.584152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242c00e-8765-4ddd-94ff-1162c0dc5ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:25.585926 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242c00e-8765-4ddd-94ff-1162c0dc5ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:25.587222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242c00e-8765-4ddd-94ff-1162c0dc5ae8 admin admin] PATH_INFO: `/v3/users/ac14029f6fdf43269fbadb2a4c65c6d1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:25.590700 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242c00e-8765-4ddd-94ff-1162c0dc5ae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac14029f6fdf43269fbadb2a4c65c6d1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:25.601904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242c00e-8765-4ddd-94ff-1162c0dc5ae8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:25.620620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a6d30-7e7c-4b29-a58b-d54798d61d48 service nova] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:25.646711 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9f9bc-6c79-4a13-b8bc-acf58a9b474a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:25.649852 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9f9bc-6c79-4a13-b8bc-acf58a9b474a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:25.651309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9f9bc-6c79-4a13-b8bc-acf58a9b474a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:25.654209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9f9bc-6c79-4a13-b8bc-acf58a9b474a admin admin] PATH_INFO: `/v3/users/d3d9710e89134d5a94719da7503a2d33` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:25.666174 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9f9bc-6c79-4a13-b8bc-acf58a9b474a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3d9710e89134d5a94719da7503a2d33)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:25.671606 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9f9bc-6c79-4a13-b8bc-acf58a9b474a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:25.902225 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3242c00e-8765-4ddd-94ff-1162c0dc5ae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac14029f6fdf43269fbadb2a4c65c6d1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:25.904129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3242c00e-8765-4ddd-94ff-1162c0dc5ae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac14029f6fdf43269fbadb2a4c65c6d1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:25.955679 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3242c00e-8765-4ddd-94ff-1162c0dc5ae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac14029f6fdf43269fbadb2a4c65c6d1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:25.959071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3242c00e-8765-4ddd-94ff-1162c0dc5ae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac14029f6fdf43269fbadb2a4c65c6d1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:26.012737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3242c00e-8765-4ddd-94ff-1162c0dc5ae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac14029f6fdf43269fbadb2a4c65c6d1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:26.112417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-40b9f9bc-6c79-4a13-b8bc-acf58a9b474a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3d9710e89134d5a94719da7503a2d33'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:26.114777 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-40b9f9bc-6c79-4a13-b8bc-acf58a9b474a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3d9710e89134d5a94719da7503a2d33'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:26.147002 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-40b9f9bc-6c79-4a13-b8bc-acf58a9b474a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3d9710e89134d5a94719da7503a2d33'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:26.155073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-40b9f9bc-6c79-4a13-b8bc-acf58a9b474a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3d9710e89134d5a94719da7503a2d33'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:26.181584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 848/1706] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:25 2022] DELETE /identity/v3/users/ac14029f6fdf43269fbadb2a4c65c6d1 => generated 0 bytes in 937 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:26.190434 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242c00e-8765-4ddd-94ff-1162c0dc5ae8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:26.196668 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3242c00e-8765-4ddd-94ff-1162c0dc5ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:26.308797 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-40b9f9bc-6c79-4a13-b8bc-acf58a9b474a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3d9710e89134d5a94719da7503a2d33'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:26.383824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242c00e-8765-4ddd-94ff-1162c0dc5ae8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:26.395774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 859/1707] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:25 2022] DELETE /identity/v3/users/d3d9710e89134d5a94719da7503a2d33 => generated 0 bytes in 1011 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:26.406069 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74134bef-fa32-4088-8eb6-e8628917a2f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:26.409684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9f9bc-6c79-4a13-b8bc-acf58a9b474a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:26.409811 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74134bef-fa32-4088-8eb6-e8628917a2f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:26.409811 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74134bef-fa32-4088-8eb6-e8628917a2f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:26.409811 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74134bef-fa32-4088-8eb6-e8628917a2f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:26.414293 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-40b9f9bc-6c79-4a13-b8bc-acf58a9b474a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:26.421235 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-74134bef-fa32-4088-8eb6-e8628917a2f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:26.670702 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74134bef-fa32-4088-8eb6-e8628917a2f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:26.673619 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74134bef-fa32-4088-8eb6-e8628917a2f4 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:26.680417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9f9bc-6c79-4a13-b8bc-acf58a9b474a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:26.692044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 849/1708] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:26 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:26.698575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fdbb50-14c6-4945-8b4d-a7fcea77d8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:26.699484 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdbb50-14c6-4945-8b4d-a7fcea77d8e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:26.699762 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdbb50-14c6-4945-8b4d-a7fcea77d8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:26.700033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdbb50-14c6-4945-8b4d-a7fcea77d8e4 admin admin] PATH_INFO: `/v3/projects/f0e5af0d10f448bdad3536aded64709a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:26.700726 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca45ed-5764-4989-99e9-655e068bb1c2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:26.701519 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca45ed-5764-4989-99e9-655e068bb1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:26.701519 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca45ed-5764-4989-99e9-655e068bb1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:26.702922 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fdbb50-14c6-4945-8b4d-a7fcea77d8e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0e5af0d10f448bdad3536aded64709a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:26.704867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fdbb50-14c6-4945-8b4d-a7fcea77d8e4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:26.720656 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:26.758457 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-79ca45ed-5764-4989-99e9-655e068bb1c2 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:26.759765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-79ca45ed-5764-4989-99e9-655e068bb1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:26.777379 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 850/1709] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:26.839275 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca45ed-5764-4989-99e9-655e068bb1c2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:26.840586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-79ca45ed-5764-4989-99e9-655e068bb1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:26.920419 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:26.937323 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-33fdbb50-14c6-4945-8b4d-a7fcea77d8e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0e5af0d10f448bdad3536aded64709a', 'request_id': 'req-33fdbb50-14c6-4945-8b4d-a7fcea77d8e4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:26.951385 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-33fdbb50-14c6-4945-8b4d-a7fcea77d8e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0e5af0d10f448bdad3536aded64709a', 'request_id': 'req-33fdbb50-14c6-4945-8b4d-a7fcea77d8e4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:27.001996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca45ed-5764-4989-99e9-655e068bb1c2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:27.044323 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-33fdbb50-14c6-4945-8b4d-a7fcea77d8e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0e5af0d10f448bdad3536aded64709a', 'request_id': 'req-33fdbb50-14c6-4945-8b4d-a7fcea77d8e4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:27.081141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-33fdbb50-14c6-4945-8b4d-a7fcea77d8e4 admin admin] The token cache is being invalidate because project f0e5af0d10f448bdad3536aded64709a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:27.081999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-33fdbb50-14c6-4945-8b4d-a7fcea77d8e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:27.083639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-33fdbb50-14c6-4945-8b4d-a7fcea77d8e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:27.100227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 860/1710] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:26 2022] DELETE /identity/v3/projects/f0e5af0d10f448bdad3536aded64709a => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:27.107418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366711cf-5028-4611-85e1-e8eaec49ecf8 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:27.118953 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366711cf-5028-4611-85e1-e8eaec49ecf8 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:27.118953 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366711cf-5028-4611-85e1-e8eaec49ecf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:27.167800 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aab1b2-93b9-4cc0-bebf-9ba5431084a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:27.167800 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aab1b2-93b9-4cc0-bebf-9ba5431084a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:27.167800 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aab1b2-93b9-4cc0-bebf-9ba5431084a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:27.179153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aab1b2-93b9-4cc0-bebf-9ba5431084a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:27.193640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-96aab1b2-93b9-4cc0-bebf-9ba5431084a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:27.227364 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-366711cf-5028-4611-85e1-e8eaec49ecf8 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:27.232376 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-366711cf-5028-4611-85e1-e8eaec49ecf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:27.409906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aab1b2-93b9-4cc0-bebf-9ba5431084a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:27.417583 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aab1b2-93b9-4cc0-bebf-9ba5431084a2 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:27.465396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 851/1711] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:38:27.592050 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aab1b2-93b9-4cc0-bebf-9ba5431084a2 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:27.593676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-96aab1b2-93b9-4cc0-bebf-9ba5431084a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:27.634511 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aab1b2-93b9-4cc0-bebf-9ba5431084a2 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:27.660056 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81b5c7-1998-4533-b035-befe7438f0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:27.661074 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 861/1712] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 559 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:27.665709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81b5c7-1998-4533-b035-befe7438f0fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:27.667515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81b5c7-1998-4533-b035-befe7438f0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:27.667904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81b5c7-1998-4533-b035-befe7438f0fc admin admin] PATH_INFO: `/v3/projects/5f754d50eb004f9e8d52207944098eb6` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:27.671400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b222fa-32a7-4de7-bbf1-4b0b0e65dcbd None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:27.671883 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b222fa-32a7-4de7-bbf1-4b0b0e65dcbd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:27.672320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b222fa-32a7-4de7-bbf1-4b0b0e65dcbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:27.676278 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81b5c7-1998-4533-b035-befe7438f0fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f754d50eb004f9e8d52207944098eb6)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:27.684685 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81b5c7-1998-4533-b035-befe7438f0fc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:27.739054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c1b222fa-32a7-4de7-bbf1-4b0b0e65dcbd None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:27.743145 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c1b222fa-32a7-4de7-bbf1-4b0b0e65dcbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:27.798511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 862/1713] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:27.798511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b222fa-32a7-4de7-bbf1-4b0b0e65dcbd None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:27.827590 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b222fa-32a7-4de7-bbf1-4b0b0e65dcbd None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:27.836400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324f856-ac02-4593-b9ad-711b2454aa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:27.837748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324f856-ac02-4593-b9ad-711b2454aa59 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:27.838536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324f856-ac02-4593-b9ad-711b2454aa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:27.839096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324f856-ac02-4593-b9ad-711b2454aa59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:27.840796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5324f856-ac02-4593-b9ad-711b2454aa59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:27.841541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5324f856-ac02-4593-b9ad-711b2454aa59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:27.848129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5324f856-ac02-4593-b9ad-711b2454aa59 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:27.861868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 863/1714] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:38:27.864969 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324f856-ac02-4593-b9ad-711b2454aa59 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:27.901288 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324f856-ac02-4593-b9ad-711b2454aa59 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:27.919616 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6e1e9-422e-43dd-9ad1-639b7e2e7dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:27.923171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6e1e9-422e-43dd-9ad1-639b7e2e7dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:27.923171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6e1e9-422e-43dd-9ad1-639b7e2e7dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:27.923171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6e1e9-422e-43dd-9ad1-639b7e2e7dd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:27.923171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6e1e9-422e-43dd-9ad1-639b7e2e7dd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:27.924070 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6e1e9-422e-43dd-9ad1-639b7e2e7dd9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:27.984868 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8e81b5c7-1998-4533-b035-befe7438f0fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f754d50eb004f9e8d52207944098eb6', 'request_id': 'req-8e81b5c7-1998-4533-b035-befe7438f0fc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:28.013961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8e81b5c7-1998-4533-b035-befe7438f0fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f754d50eb004f9e8d52207944098eb6', 'request_id': 'req-8e81b5c7-1998-4533-b035-befe7438f0fc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:28.030606 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8e81b5c7-1998-4533-b035-befe7438f0fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f754d50eb004f9e8d52207944098eb6', 'request_id': 'req-8e81b5c7-1998-4533-b035-befe7438f0fc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:28.070721 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 864/1715] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:27 2022] POST /identity/v3/projects => generated 371 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:38:28.075383 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6e1e9-422e-43dd-9ad1-639b7e2e7dd9 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:28.141474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8e81b5c7-1998-4533-b035-befe7438f0fc admin admin] The token cache is being invalidate because project 5f754d50eb004f9e8d52207944098eb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:28.141843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8e81b5c7-1998-4533-b035-befe7438f0fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:28.142644 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8e81b5c7-1998-4533-b035-befe7438f0fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:28.147526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 852/1716] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:27 2022] DELETE /identity/v3/projects/5f754d50eb004f9e8d52207944098eb6 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:28.152791 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5ed22-85a8-48a9-ab49-6e86672f1cae None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:28.153153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5ed22-85a8-48a9-ab49-6e86672f1cae None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:28.153386 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5ed22-85a8-48a9-ab49-6e86672f1cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:28.289409 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b2d5ed22-85a8-48a9-ab49-6e86672f1cae None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:28.302319 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b2d5ed22-85a8-48a9-ab49-6e86672f1cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:28.428210 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6e1e9-422e-43dd-9ad1-639b7e2e7dd9 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:28.460779 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea838d-4ab2-4f5d-84ff-de6990bbc0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:28.464387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea838d-4ab2-4f5d-84ff-de6990bbc0a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:28.464540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea838d-4ab2-4f5d-84ff-de6990bbc0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:28.464884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea838d-4ab2-4f5d-84ff-de6990bbc0a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:28.468032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea838d-4ab2-4f5d-84ff-de6990bbc0a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:28.476616 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea838d-4ab2-4f5d-84ff-de6990bbc0a5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:28.501977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 853/1717] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:28.503955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f9fe8-cb9b-40b6-ae91-e19910c69a60 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:28.504622 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f9fe8-cb9b-40b6-ae91-e19910c69a60 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:28.505155 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f9fe8-cb9b-40b6-ae91-e19910c69a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:28.551933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-033f9fe8-cb9b-40b6-ae91-e19910c69a60 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:28.554263 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-033f9fe8-cb9b-40b6-ae91-e19910c69a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:28.601748 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 854/1718] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:28.602765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f9fe8-cb9b-40b6-ae91-e19910c69a60 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:28.622792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f9fe8-cb9b-40b6-ae91-e19910c69a60 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:28.628399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 865/1719] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:28 2022] POST /identity/v3/users => generated 286 bytes in 553 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:28.633880 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea838d-4ab2-4f5d-84ff-de6990bbc0a5 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:28.635735 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-efea838d-4ab2-4f5d-84ff-de6990bbc0a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:28.640042 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e37a21-1312-4fa2-9487-ec0c2b28ec0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:28.641255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e37a21-1312-4fa2-9487-ec0c2b28ec0c admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:28.641772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e37a21-1312-4fa2-9487-ec0c2b28ec0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:28.642240 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e37a21-1312-4fa2-9487-ec0c2b28ec0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:28.643187 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e37a21-1312-4fa2-9487-ec0c2b28ec0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:28.643693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e37a21-1312-4fa2-9487-ec0c2b28ec0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:28.647858 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e37a21-1312-4fa2-9487-ec0c2b28ec0c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:28.696569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 855/1720] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:28.701860 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e37a21-1312-4fa2-9487-ec0c2b28ec0c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:28.703549 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea838d-4ab2-4f5d-84ff-de6990bbc0a5 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:28.719067 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a22e6-bd0d-49e9-89f0-3853f1344148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:28.722996 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a22e6-bd0d-49e9-89f0-3853f1344148 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:28.727977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a22e6-bd0d-49e9-89f0-3853f1344148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:28.727977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a22e6-bd0d-49e9-89f0-3853f1344148 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:28.728145 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e37a21-1312-4fa2-9487-ec0c2b28ec0c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:28.728233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a22e6-bd0d-49e9-89f0-3853f1344148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:28.730272 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a22e6-bd0d-49e9-89f0-3853f1344148 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:28.748215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7754e-d5d0-42ab-98d7-2f69427a7f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:28.751395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7754e-d5d0-42ab-98d7-2f69427a7f88 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:28.751758 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7754e-d5d0-42ab-98d7-2f69427a7f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:28.752011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7754e-d5d0-42ab-98d7-2f69427a7f88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:28.756598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7754e-d5d0-42ab-98d7-2f69427a7f88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:28.756598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7754e-d5d0-42ab-98d7-2f69427a7f88 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:28.759623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 866/1721] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:38:28.765690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a22e6-bd0d-49e9-89f0-3853f1344148 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:28.814238 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a22e6-bd0d-49e9-89f0-3853f1344148 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:28.847104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f00114-6f42-4079-a588-81d6b2c15274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:28.848258 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f00114-6f42-4079-a588-81d6b2c15274 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:28.848630 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f00114-6f42-4079-a588-81d6b2c15274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:28.848910 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f00114-6f42-4079-a588-81d6b2c15274 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:28.856819 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f00114-6f42-4079-a588-81d6b2c15274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:28.862005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f00114-6f42-4079-a588-81d6b2c15274 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:28.884933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 867/1722] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:28.896228 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f00114-6f42-4079-a588-81d6b2c15274 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:28.924911 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f00114-6f42-4079-a588-81d6b2c15274 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:28.956607 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9532f69-6a50-41da-9b08-0448079687da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:28.957760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9532f69-6a50-41da-9b08-0448079687da admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:28.958674 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9532f69-6a50-41da-9b08-0448079687da admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:28.959811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9532f69-6a50-41da-9b08-0448079687da admin admin] PATH_INFO: `/v3/projects/fb4a6b81990b40ac958b028bff582296/users/89473e52c78240e88665e4b1c487cd12/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:29.010280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 856/1723] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:28 2022] POST /identity/v3/projects => generated 385 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:29.016251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7754e-d5d0-42ab-98d7-2f69427a7f88 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:29.029498 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9532f69-6a50-41da-9b08-0448079687da admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb4a6b81990b40ac958b028bff582296, user_id=89473e52c78240e88665e4b1c487cd12, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:29.043856 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9532f69-6a50-41da-9b08-0448079687da admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:29.175483 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7754e-d5d0-42ab-98d7-2f69427a7f88 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:29.198628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 868/1724] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:28 2022] PUT /identity/v3/projects/fb4a6b81990b40ac958b028bff582296/users/89473e52c78240e88665e4b1c487cd12/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:29.209456 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90438163-7534-488a-b58f-4bea68d6555f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:29.209769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90438163-7534-488a-b58f-4bea68d6555f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:29.210169 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90438163-7534-488a-b58f-4bea68d6555f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:29.310889 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-90438163-7534-488a-b58f-4bea68d6555f None None] MFA Rules not processed for user `89473e52c78240e88665e4b1c487cd12`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:29.328917 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-90438163-7534-488a-b58f-4bea68d6555f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:29.340820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e161c4-5f40-4efa-870c-aaf6ba6e413b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:29.341620 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e161c4-5f40-4efa-870c-aaf6ba6e413b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:29.360786 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e161c4-5f40-4efa-870c-aaf6ba6e413b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:29.360786 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e161c4-5f40-4efa-870c-aaf6ba6e413b admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:29.361398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e161c4-5f40-4efa-870c-aaf6ba6e413b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:29.363912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e161c4-5f40-4efa-870c-aaf6ba6e413b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:29.471849 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 857/1725] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:29 2022] POST /identity/v3/users => generated 293 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:29.476902 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e161c4-5f40-4efa-870c-aaf6ba6e413b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:29.499246 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e161c4-5f40-4efa-870c-aaf6ba6e413b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:29.509719 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94f0f4-85f4-4f37-9f55-b97aeb19b9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:29.511737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94f0f4-85f4-4f37-9f55-b97aeb19b9a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:29.513642 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94f0f4-85f4-4f37-9f55-b97aeb19b9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:29.514867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94f0f4-85f4-4f37-9f55-b97aeb19b9a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:29.515838 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94f0f4-85f4-4f37-9f55-b97aeb19b9a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:29.519806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94f0f4-85f4-4f37-9f55-b97aeb19b9a2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:29.530220 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 869/1726] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:29.540625 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90438163-7534-488a-b58f-4bea68d6555f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:29.543511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-90438163-7534-488a-b58f-4bea68d6555f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:29.589034 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 858/1727] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:29.594689 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94f0f4-85f4-4f37-9f55-b97aeb19b9a2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:29.622937 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94f0f4-85f4-4f37-9f55-b97aeb19b9a2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:29.641273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf854c8-b46f-4b0f-a207-91a41dad3122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:29.642226 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf854c8-b46f-4b0f-a207-91a41dad3122 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:29.642683 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf854c8-b46f-4b0f-a207-91a41dad3122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:29.643155 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf854c8-b46f-4b0f-a207-91a41dad3122 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:29.644053 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf854c8-b46f-4b0f-a207-91a41dad3122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:29.646162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf854c8-b46f-4b0f-a207-91a41dad3122 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:29.680747 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 859/1728] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:29.685865 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf854c8-b46f-4b0f-a207-91a41dad3122 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:29.710931 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf854c8-b46f-4b0f-a207-91a41dad3122 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:29.726576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3921e-298d-4810-9a53-3e67ec2ecfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:29.730638 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3921e-298d-4810-9a53-3e67ec2ecfd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:29.732492 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3921e-298d-4810-9a53-3e67ec2ecfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:29.733034 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3921e-298d-4810-9a53-3e67ec2ecfd3 admin admin] PATH_INFO: `/v3/projects/3c658ffe76a24bf78d42c6458a482ee6/users/6cc36c588f3441698a50c4632f145636/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:29.786243 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3921e-298d-4810-9a53-3e67ec2ecfd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c658ffe76a24bf78d42c6458a482ee6, user_id=6cc36c588f3441698a50c4632f145636, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:29.798541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3921e-298d-4810-9a53-3e67ec2ecfd3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:29.860534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90438163-7534-488a-b58f-4bea68d6555f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:29.873789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115471e-46d9-461b-9881-caff616c1e3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:29.874639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115471e-46d9-461b-9881-caff616c1e3c service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:29.875872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115471e-46d9-461b-9881-caff616c1e3c service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:29.876202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115471e-46d9-461b-9881-caff616c1e3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:29.895187 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115471e-46d9-461b-9881-caff616c1e3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:29.897975 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115471e-46d9-461b-9881-caff616c1e3c service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:29.919864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 860/1729] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:29 2022] PUT /identity/v3/projects/3c658ffe76a24bf78d42c6458a482ee6/users/6cc36c588f3441698a50c4632f145636/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:29.921272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd900373-dba6-43db-aab7-e149cf43d3e9 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:29.921562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd900373-dba6-43db-aab7-e149cf43d3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:29.921844 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd900373-dba6-43db-aab7-e149cf43d3e9 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:29.923175 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 861/1730] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jun 7 08:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:38:29.941559 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98480392-ba0e-458c-bd79-5ff09e26ff28 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:29.941898 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98480392-ba0e-458c-bd79-5ff09e26ff28 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:29.942130 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98480392-ba0e-458c-bd79-5ff09e26ff28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:29.946923 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 870/1731] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:30.002295 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-98480392-ba0e-458c-bd79-5ff09e26ff28 None None] MFA Rules not processed for user `6cc36c588f3441698a50c4632f145636`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:30.003658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-98480392-ba0e-458c-bd79-5ff09e26ff28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:30.128524 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fead9ca-3b80-4447-84d2-c82a0f98b589 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:30.128524 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fead9ca-3b80-4447-84d2-c82a0f98b589 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:30.128524 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fead9ca-3b80-4447-84d2-c82a0f98b589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:30.162995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-4fead9ca-3b80-4447-84d2-c82a0f98b589 None None] MFA Rules not processed for user `ee42568b059b49a198ed2c01d5efdafd`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:30.164238 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4fead9ca-3b80-4447-84d2-c82a0f98b589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:30.235812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:30.254785 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 862/1732] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:30.258923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98480392-ba0e-458c-bd79-5ff09e26ff28 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:30.458658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:30.479251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98480392-ba0e-458c-bd79-5ff09e26ff28 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:30.499876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147709e-a9cd-423e-a8f6-c091bb12c905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:30.500598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147709e-a9cd-423e-a8f6-c091bb12c905 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:30.500910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147709e-a9cd-423e-a8f6-c091bb12c905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:30.501249 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147709e-a9cd-423e-a8f6-c091bb12c905 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:30.502744 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147709e-a9cd-423e-a8f6-c091bb12c905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:30.504944 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147709e-a9cd-423e-a8f6-c091bb12c905 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:30.608642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 871/1733] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jun 7 08:38:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 483 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:38:30.660111 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fead9ca-3b80-4447-84d2-c82a0f98b589 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:30.661600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4fead9ca-3b80-4447-84d2-c82a0f98b589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:30.703401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 863/1734] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:30 2022] POST /identity/v3/projects => generated 387 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:30.705209 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147709e-a9cd-423e-a8f6-c091bb12c905 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:30.706079 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fead9ca-3b80-4447-84d2-c82a0f98b589 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:30.959447 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147709e-a9cd-423e-a8f6-c091bb12c905 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:30.960323 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b59734-88f2-4f00-ab1d-d9640f3bb8b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:30.961753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b59734-88f2-4f00-ab1d-d9640f3bb8b3 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:30.963071 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b59734-88f2-4f00-ab1d-d9640f3bb8b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:30.963504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b59734-88f2-4f00-ab1d-d9640f3bb8b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:30.974270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185d429-3e57-4144-86c4-2dc06f7b16f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:30.976333 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185d429-3e57-4144-86c4-2dc06f7b16f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:30.976829 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185d429-3e57-4144-86c4-2dc06f7b16f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:30.977396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185d429-3e57-4144-86c4-2dc06f7b16f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:30.978820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185d429-3e57-4144-86c4-2dc06f7b16f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:30.987562 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b59734-88f2-4f00-ab1d-d9640f3bb8b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:30.992681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b59734-88f2-4f00-ab1d-d9640f3bb8b3 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:30.993997 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185d429-3e57-4144-86c4-2dc06f7b16f5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:31.018719 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 872/1735] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:38:31.020222 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b59734-88f2-4f00-ab1d-d9640f3bb8b3 service nova] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:31.022756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-70b59734-88f2-4f00-ab1d-d9640f3bb8b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:31.056394 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b59734-88f2-4f00-ab1d-d9640f3bb8b3 service nova] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:31.085534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0dac2-7418-47b2-b590-0c14a2f54252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:31.086983 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0dac2-7418-47b2-b590-0c14a2f54252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:31.087479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0dac2-7418-47b2-b590-0c14a2f54252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:31.088210 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0dac2-7418-47b2-b590-0c14a2f54252 admin admin] PATH_INFO: `/v3/projects/b371f1c3f8914cf4898d65538657bb57` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:31.093724 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0dac2-7418-47b2-b590-0c14a2f54252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b371f1c3f8914cf4898d65538657bb57)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:31.100277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0dac2-7418-47b2-b590-0c14a2f54252 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:31.135181 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 864/1736] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:30 2022] POST /identity/v3/users => generated 300 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:31.137401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185d429-3e57-4144-86c4-2dc06f7b16f5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:31.204867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185d429-3e57-4144-86c4-2dc06f7b16f5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:31.230934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f777858-7a69-4657-ba64-1f47828f8edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:31.232121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f777858-7a69-4657-ba64-1f47828f8edd admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:31.232595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f777858-7a69-4657-ba64-1f47828f8edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:31.233075 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f777858-7a69-4657-ba64-1f47828f8edd admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:31.234463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f777858-7a69-4657-ba64-1f47828f8edd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:31.249412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f777858-7a69-4657-ba64-1f47828f8edd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:31.301685 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 865/1737] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:31.303262 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f777858-7a69-4657-ba64-1f47828f8edd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:31.394672 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f777858-7a69-4657-ba64-1f47828f8edd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:31.413775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bec0dac2-7418-47b2-b590-0c14a2f54252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b371f1c3f8914cf4898d65538657bb57', 'request_id': 'req-bec0dac2-7418-47b2-b590-0c14a2f54252'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:31.441056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bec0dac2-7418-47b2-b590-0c14a2f54252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b371f1c3f8914cf4898d65538657bb57', 'request_id': 'req-bec0dac2-7418-47b2-b590-0c14a2f54252'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:31.559374 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bec0dac2-7418-47b2-b590-0c14a2f54252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b371f1c3f8914cf4898d65538657bb57', 'request_id': 'req-bec0dac2-7418-47b2-b590-0c14a2f54252'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:31.579147 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bec0dac2-7418-47b2-b590-0c14a2f54252 admin admin] The token cache is being invalidate because project b371f1c3f8914cf4898d65538657bb57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:31.579610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bec0dac2-7418-47b2-b590-0c14a2f54252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:31.580343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bec0dac2-7418-47b2-b590-0c14a2f54252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:31.582460 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 873/1738] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:31 2022] DELETE /identity/v3/projects/b371f1c3f8914cf4898d65538657bb57 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:31.609511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6e30c-e1e2-4018-bcc9-08c928e741ec None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:31.620563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6e30c-e1e2-4018-bcc9-08c928e741ec None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:31.622030 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6e30c-e1e2-4018-bcc9-08c928e741ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:31.732877 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-05b6e30c-e1e2-4018-bcc9-08c928e741ec None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:31.734127 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-05b6e30c-e1e2-4018-bcc9-08c928e741ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:31.760010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000e9aa-8eb5-4bcd-9aac-ff71f70a3159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:31.760813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000e9aa-8eb5-4bcd-9aac-ff71f70a3159 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:31.761120 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000e9aa-8eb5-4bcd-9aac-ff71f70a3159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:31.777482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000e9aa-8eb5-4bcd-9aac-ff71f70a3159 admin admin] PATH_INFO: `/v3/projects/17e12cf13f1749baac36fd2ac82d326d/users/0f2fd299c7544351b6d5e938b1180706/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:31.796758 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 874/1739] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:31.805530 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46675f89-34e8-450e-a844-16f6d2806c3f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:31.805861 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46675f89-34e8-450e-a844-16f6d2806c3f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:31.808439 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46675f89-34e8-450e-a844-16f6d2806c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:31.891762 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b000e9aa-8eb5-4bcd-9aac-ff71f70a3159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17e12cf13f1749baac36fd2ac82d326d, user_id=0f2fd299c7544351b6d5e938b1180706, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:31.897695 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b000e9aa-8eb5-4bcd-9aac-ff71f70a3159 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:31.955385 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-46675f89-34e8-450e-a844-16f6d2806c3f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:31.958841 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-46675f89-34e8-450e-a844-16f6d2806c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:32.001694 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 875/1740] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:32.006831 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46675f89-34e8-450e-a844-16f6d2806c3f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:32.025767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 866/1741] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:31 2022] PUT /identity/v3/projects/17e12cf13f1749baac36fd2ac82d326d/users/0f2fd299c7544351b6d5e938b1180706/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:32.030997 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000e9aa-8eb5-4bcd-9aac-ff71f70a3159 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:32.035699 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b000e9aa-8eb5-4bcd-9aac-ff71f70a3159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:32.062165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46675f89-34e8-450e-a844-16f6d2806c3f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:32.254873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c266353-c896-4527-a2e1-e3f71e5a5bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:32.255559 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c266353-c896-4527-a2e1-e3f71e5a5bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:32.255841 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c266353-c896-4527-a2e1-e3f71e5a5bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:32.256208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c266353-c896-4527-a2e1-e3f71e5a5bd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:32.257025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c266353-c896-4527-a2e1-e3f71e5a5bd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:32.257378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c266353-c896-4527-a2e1-e3f71e5a5bd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:32.259168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c266353-c896-4527-a2e1-e3f71e5a5bd1 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:32.365066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000e9aa-8eb5-4bcd-9aac-ff71f70a3159 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:32.366737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 876/1742] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:32.376463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c266353-c896-4527-a2e1-e3f71e5a5bd1 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:32.395197 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee155a-492a-461c-8a68-2bc771c63b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:32.396134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee155a-492a-461c-8a68-2bc771c63b0a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:32.398151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee155a-492a-461c-8a68-2bc771c63b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:32.398616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee155a-492a-461c-8a68-2bc771c63b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:32.399507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee155a-492a-461c-8a68-2bc771c63b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:32.410781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee155a-492a-461c-8a68-2bc771c63b0a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:32.429384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c266353-c896-4527-a2e1-e3f71e5a5bd1 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:32.448974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 867/1743] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:32.450255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee155a-492a-461c-8a68-2bc771c63b0a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:32.483676 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b6258-899f-4eff-8200-8528bd3a137c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:32.489439 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee155a-492a-461c-8a68-2bc771c63b0a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:32.494544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b6258-899f-4eff-8200-8528bd3a137c admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:32.495269 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b6258-899f-4eff-8200-8528bd3a137c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:32.495721 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b6258-899f-4eff-8200-8528bd3a137c admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:32.497028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b6258-899f-4eff-8200-8528bd3a137c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:32.500535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b6258-899f-4eff-8200-8528bd3a137c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:32.501565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267dd503-d9a9-43ff-b787-8ddbb6ab5d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:32.502939 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267dd503-d9a9-43ff-b787-8ddbb6ab5d65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:32.503367 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267dd503-d9a9-43ff-b787-8ddbb6ab5d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:32.503892 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267dd503-d9a9-43ff-b787-8ddbb6ab5d65 admin admin] PATH_INFO: `/v3/projects/17e12cf13f1749baac36fd2ac82d326d/users/0f2fd299c7544351b6d5e938b1180706/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:32.507799 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267dd503-d9a9-43ff-b787-8ddbb6ab5d65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17e12cf13f1749baac36fd2ac82d326d, user_id=0f2fd299c7544351b6d5e938b1180706, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:32.511889 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267dd503-d9a9-43ff-b787-8ddbb6ab5d65 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:32.545139 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 868/1744] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:32 2022] PUT /identity/v3/projects/17e12cf13f1749baac36fd2ac82d326d/users/0f2fd299c7544351b6d5e938b1180706/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:32.561827 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df578933-0d8c-42e1-b8ba-0d1b7bf4e540 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:32.562257 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df578933-0d8c-42e1-b8ba-0d1b7bf4e540 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:32.562678 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df578933-0d8c-42e1-b8ba-0d1b7bf4e540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:32.644815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-df578933-0d8c-42e1-b8ba-0d1b7bf4e540 None None] MFA Rules not processed for user `0f2fd299c7544351b6d5e938b1180706`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:32.646404 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-df578933-0d8c-42e1-b8ba-0d1b7bf4e540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:32.696740 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:32.773867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 877/1745] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:32 2022] POST /identity/v3/projects => generated 405 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:32.781796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b6258-899f-4eff-8200-8528bd3a137c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:32.868946 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 869/1746] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:32.872440 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:32.999593 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df578933-0d8c-42e1-b8ba-0d1b7bf4e540 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:33.002131 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-df578933-0d8c-42e1-b8ba-0d1b7bf4e540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:33.013860 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b6258-899f-4eff-8200-8528bd3a137c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:33.034001 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd90d4c-fd70-4b64-ba92-0b69d3ce6681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:33.035356 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd90d4c-fd70-4b64-ba92-0b69d3ce6681 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:33.035904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd90d4c-fd70-4b64-ba92-0b69d3ce6681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:33.036564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd90d4c-fd70-4b64-ba92-0b69d3ce6681 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:33.037514 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd90d4c-fd70-4b64-ba92-0b69d3ce6681 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:33.043659 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd90d4c-fd70-4b64-ba92-0b69d3ce6681 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:33.127267 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df578933-0d8c-42e1-b8ba-0d1b7bf4e540 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:33.202790 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 878/1747] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:32 2022] POST /identity/v3/users => generated 303 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:33.203691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd90d4c-fd70-4b64-ba92-0b69d3ce6681 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:33.231741 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd90d4c-fd70-4b64-ba92-0b69d3ce6681 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:33.242047 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc07d1-a449-441e-9399-40287321ff7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:33.246798 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc07d1-a449-441e-9399-40287321ff7e admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:33.249271 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043800a-db93-4f89-a4d7-7adf25ebf135 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:33.250306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc07d1-a449-441e-9399-40287321ff7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:33.251140 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043800a-db93-4f89-a4d7-7adf25ebf135 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:33.251684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc07d1-a449-441e-9399-40287321ff7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:33.252188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043800a-db93-4f89-a4d7-7adf25ebf135 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:33.253158 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc07d1-a449-441e-9399-40287321ff7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:33.253636 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043800a-db93-4f89-a4d7-7adf25ebf135 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:33.256069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc07d1-a449-441e-9399-40287321ff7e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:33.269460 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 879/1748] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:33.273240 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc07d1-a449-441e-9399-40287321ff7e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:33.287202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc07d1-a449-441e-9399-40287321ff7e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:33.293227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2ae95-c1a9-4c5f-b4c3-474d56d9643c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:33.294394 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2ae95-c1a9-4c5f-b4c3-474d56d9643c admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:33.294779 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2ae95-c1a9-4c5f-b4c3-474d56d9643c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:33.295281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2ae95-c1a9-4c5f-b4c3-474d56d9643c admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:33.296262 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2ae95-c1a9-4c5f-b4c3-474d56d9643c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:33.298717 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2ae95-c1a9-4c5f-b4c3-474d56d9643c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:33.314734 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 880/1749] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:33.318684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2ae95-c1a9-4c5f-b4c3-474d56d9643c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:33.340310 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2ae95-c1a9-4c5f-b4c3-474d56d9643c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:33.347880 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730162bf-7e7b-42b0-814d-fbee1731d05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:33.348950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730162bf-7e7b-42b0-814d-fbee1731d05e admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:33.349597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730162bf-7e7b-42b0-814d-fbee1731d05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:33.349977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730162bf-7e7b-42b0-814d-fbee1731d05e admin admin] PATH_INFO: `/v3/projects/017d2b198d154b2181ea4da8a8d9ea54/users/eb90c9b7550b4b17bf5d228582d2dcb1/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:33.371049 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730162bf-7e7b-42b0-814d-fbee1731d05e admin admin] RBAC: Authorizing `identity:create_grant(project_id=017d2b198d154b2181ea4da8a8d9ea54, user_id=eb90c9b7550b4b17bf5d228582d2dcb1, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:33.374035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730162bf-7e7b-42b0-814d-fbee1731d05e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:33.399717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043800a-db93-4f89-a4d7-7adf25ebf135 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:33.401765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043800a-db93-4f89-a4d7-7adf25ebf135 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:33.413429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 870/1750] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:33.437459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 881/1751] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:33 2022] PUT /identity/v3/projects/017d2b198d154b2181ea4da8a8d9ea54/users/eb90c9b7550b4b17bf5d228582d2dcb1/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:38:33.469505 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc6541-075f-4f89-a2a5-6d09d28fd192 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:33.472065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc6541-075f-4f89-a2a5-6d09d28fd192 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:33.472409 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc6541-075f-4f89-a2a5-6d09d28fd192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:33.576171 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-74bc6541-075f-4f89-a2a5-6d09d28fd192 None None] MFA Rules not processed for user `eb90c9b7550b4b17bf5d228582d2dcb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:33.579258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-74bc6541-075f-4f89-a2a5-6d09d28fd192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:33.670009 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730162bf-7e7b-42b0-814d-fbee1731d05e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:33.726672 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 871/1752] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:33 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:33.728011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc6541-075f-4f89-a2a5-6d09d28fd192 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:33.850188 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730162bf-7e7b-42b0-814d-fbee1731d05e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:33.858322 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807a36b-a42a-4f55-8c14-592dba7dfb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:33.860443 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807a36b-a42a-4f55-8c14-592dba7dfb90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:33.860737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807a36b-a42a-4f55-8c14-592dba7dfb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:33.861126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807a36b-a42a-4f55-8c14-592dba7dfb90 admin admin] PATH_INFO: `/v3/users/6cc36c588f3441698a50c4632f145636` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:33.863025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a807a36b-a42a-4f55-8c14-592dba7dfb90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cc36c588f3441698a50c4632f145636)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:33.865703 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a807a36b-a42a-4f55-8c14-592dba7dfb90 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:33.931282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc6541-075f-4f89-a2a5-6d09d28fd192 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:33.955722 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f99b6b-f7d9-4f7a-9dce-64b8e72cf8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:33.960715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f99b6b-f7d9-4f7a-9dce-64b8e72cf8dd admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:33.960715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f99b6b-f7d9-4f7a-9dce-64b8e72cf8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:33.960715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f99b6b-f7d9-4f7a-9dce-64b8e72cf8dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:33.960715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f99b6b-f7d9-4f7a-9dce-64b8e72cf8dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:33.966652 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f99b6b-f7d9-4f7a-9dce-64b8e72cf8dd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:34.090165 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 872/1753] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:33 2022] POST /identity/v3/projects => generated 405 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:34.097439 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:34.098919 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f99b6b-f7d9-4f7a-9dce-64b8e72cf8dd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:34.120877 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a807a36b-a42a-4f55-8c14-592dba7dfb90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cc36c588f3441698a50c4632f145636'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:34.122108 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a807a36b-a42a-4f55-8c14-592dba7dfb90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cc36c588f3441698a50c4632f145636'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:34.161674 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a807a36b-a42a-4f55-8c14-592dba7dfb90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cc36c588f3441698a50c4632f145636'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:34.164291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a807a36b-a42a-4f55-8c14-592dba7dfb90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cc36c588f3441698a50c4632f145636'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:34.243508 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a807a36b-a42a-4f55-8c14-592dba7dfb90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cc36c588f3441698a50c4632f145636'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:34.294089 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f99b6b-f7d9-4f7a-9dce-64b8e72cf8dd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:34.305677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 882/1754] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:33 2022] DELETE /identity/v3/users/6cc36c588f3441698a50c4632f145636 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:34.307312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe4d6b-2deb-42ae-ac99-5a9e4a1e90dd None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:34.307598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe4d6b-2deb-42ae-ac99-5a9e4a1e90dd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:34.307877 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe4d6b-2deb-42ae-ac99-5a9e4a1e90dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:34.374197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c3fe4d6b-2deb-42ae-ac99-5a9e4a1e90dd None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:34.379334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c3fe4d6b-2deb-42ae-ac99-5a9e4a1e90dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:34.414209 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55add7b3-4971-4487-8b3e-84bd821f754e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:34.414798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55add7b3-4971-4487-8b3e-84bd821f754e admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:34.414798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55add7b3-4971-4487-8b3e-84bd821f754e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:34.415053 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55add7b3-4971-4487-8b3e-84bd821f754e admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:34.415751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55add7b3-4971-4487-8b3e-84bd821f754e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:34.417739 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55add7b3-4971-4487-8b3e-84bd821f754e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:34.496644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 883/1755] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:38:34.513595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 873/1756] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:34 2022] POST /identity/v3/users => generated 309 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:34.517246 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe4d6b-2deb-42ae-ac99-5a9e4a1e90dd None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:34.519286 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c3fe4d6b-2deb-42ae-ac99-5a9e4a1e90dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:34.525327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55add7b3-4971-4487-8b3e-84bd821f754e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:34.527406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-55add7b3-4971-4487-8b3e-84bd821f754e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:34.564707 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe4d6b-2deb-42ae-ac99-5a9e4a1e90dd None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:34.582716 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa7d6d-0bd0-4297-a975-169bbfae9c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:34.583410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa7d6d-0bd0-4297-a975-169bbfae9c1e admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:34.583687 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa7d6d-0bd0-4297-a975-169bbfae9c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:34.583963 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa7d6d-0bd0-4297-a975-169bbfae9c1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:34.586741 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa7d6d-0bd0-4297-a975-169bbfae9c1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:34.591718 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa7d6d-0bd0-4297-a975-169bbfae9c1e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:34.618881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 884/1757] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:34.622523 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c5e99-1b78-4c2e-a012-010a36b4bcb7 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:34.622863 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c5e99-1b78-4c2e-a012-010a36b4bcb7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:34.623144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c5e99-1b78-4c2e-a012-010a36b4bcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:34.623588 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:34.655463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-8c1c5e99-1b78-4c2e-a012-010a36b4bcb7 None None] MFA Rules not processed for user `0b2efb9857cf495f8b51a96b5f868d44`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:34.657300 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8c1c5e99-1b78-4c2e-a012-010a36b4bcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:34.683239 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55add7b3-4971-4487-8b3e-84bd821f754e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:34.694077 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2a3f3-ecda-408f-bf4c-48e4ee75a89e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:34.694753 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2a3f3-ecda-408f-bf4c-48e4ee75a89e service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:34.695039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2a3f3-ecda-408f-bf4c-48e4ee75a89e service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:34.695311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2a3f3-ecda-408f-bf4c-48e4ee75a89e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:34.710338 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2a3f3-ecda-408f-bf4c-48e4ee75a89e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:34.712092 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2a3f3-ecda-408f-bf4c-48e4ee75a89e service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:34.734009 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 874/1758] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:38:34.734973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2a3f3-ecda-408f-bf4c-48e4ee75a89e service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:34.758707 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 885/1759] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:34.762979 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2a3f3-ecda-408f-bf4c-48e4ee75a89e service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:34.786675 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48d960-d7d6-44ea-8291-5c4f7f2f30ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:34.787645 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48d960-d7d6-44ea-8291-5c4f7f2f30ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:34.788604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48d960-d7d6-44ea-8291-5c4f7f2f30ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:34.788882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48d960-d7d6-44ea-8291-5c4f7f2f30ee admin admin] PATH_INFO: `/v3/projects/5128fff40ebd431fb51cff07772b17bb/users/9d296aa5db0c4dc5a2799bd9bf64440e/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:34.836929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48d960-d7d6-44ea-8291-5c4f7f2f30ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=5128fff40ebd431fb51cff07772b17bb, user_id=9d296aa5db0c4dc5a2799bd9bf64440e, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:34.840125 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48d960-d7d6-44ea-8291-5c4f7f2f30ee admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:34.965824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 875/1760] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:34 2022] PUT /identity/v3/projects/5128fff40ebd431fb51cff07772b17bb/users/9d296aa5db0c4dc5a2799bd9bf64440e/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:34.971166 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c5e99-1b78-4c2e-a012-010a36b4bcb7 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:35.007615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c5e99-1b78-4c2e-a012-010a36b4bcb7 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:35.156981 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2d0c2-b5af-4680-8098-597f0a00f278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:35.157953 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2d0c2-b5af-4680-8098-597f0a00f278 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:35.158105 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2d0c2-b5af-4680-8098-597f0a00f278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:35.158455 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2d0c2-b5af-4680-8098-597f0a00f278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:35.159239 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2d0c2-b5af-4680-8098-597f0a00f278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:35.170789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af90d13-4b8c-446e-9e80-7d81e4ab97cd None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:35.171102 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af90d13-4b8c-446e-9e80-7d81e4ab97cd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:35.171388 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af90d13-4b8c-446e-9e80-7d81e4ab97cd None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:35.172380 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2d0c2-b5af-4680-8098-597f0a00f278 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:35.173163 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 876/1761] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jun 7 08:38:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:38:35.196762 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 886/1762] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:35.201474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af90d13-4b8c-446e-9e80-7d81e4ab97cd None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:35.228374 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af90d13-4b8c-446e-9e80-7d81e4ab97cd None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:35.234477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45408c-c891-45f0-98cb-b2dcdb73f604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:35.246457 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5e1dc-40b5-4cfe-8ab6-377e48526bca None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:35.247994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5e1dc-40b5-4cfe-8ab6-377e48526bca None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:35.248364 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5e1dc-40b5-4cfe-8ab6-377e48526bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:35.253546 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45408c-c891-45f0-98cb-b2dcdb73f604 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:35.253546 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45408c-c891-45f0-98cb-b2dcdb73f604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:35.253546 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45408c-c891-45f0-98cb-b2dcdb73f604 admin admin] PATH_INFO: `/v3/projects/5128fff40ebd431fb51cff07772b17bb/users/9d296aa5db0c4dc5a2799bd9bf64440e/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:35.260835 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45408c-c891-45f0-98cb-b2dcdb73f604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5128fff40ebd431fb51cff07772b17bb, user_id=9d296aa5db0c4dc5a2799bd9bf64440e, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:35.264903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45408c-c891-45f0-98cb-b2dcdb73f604 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:35.281563 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 877/1763] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:35 2022] PUT /identity/v3/projects/5128fff40ebd431fb51cff07772b17bb/users/9d296aa5db0c4dc5a2799bd9bf64440e/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:35.304790 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e9a3c-9b63-42d4-888d-d96c4de0cb48 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:35.305620 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e9a3c-9b63-42d4-888d-d96c4de0cb48 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:35.306042 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e9a3c-9b63-42d4-888d-d96c4de0cb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:35.306152 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d8e5e1dc-40b5-4cfe-8ab6-377e48526bca None None] MFA Rules not processed for user `ee42568b059b49a198ed2c01d5efdafd`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:35.307413 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d8e5e1dc-40b5-4cfe-8ab6-377e48526bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:35.355506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ab2e9a3c-9b63-42d4-888d-d96c4de0cb48 None None] MFA Rules not processed for user `9d296aa5db0c4dc5a2799bd9bf64440e`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:35.357921 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ab2e9a3c-9b63-42d4-888d-d96c4de0cb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:35.471895 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:35.525597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:35.567791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 887/1764] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jun 7 08:38:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:35.589394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 878/1765] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:35 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:35.599290 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5e1dc-40b5-4cfe-8ab6-377e48526bca None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:35.601498 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d8e5e1dc-40b5-4cfe-8ab6-377e48526bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:35.609274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e9a3c-9b63-42d4-888d-d96c4de0cb48 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:35.611137 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ab2e9a3c-9b63-42d4-888d-d96c4de0cb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:35.632190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e9a3c-9b63-42d4-888d-d96c4de0cb48 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:35.649029 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7281d-4a34-4b79-af8f-9c5db6f0ae8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:35.650085 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7281d-4a34-4b79-af8f-9c5db6f0ae8f service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:35.650575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7281d-4a34-4b79-af8f-9c5db6f0ae8f service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:35.651080 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7281d-4a34-4b79-af8f-9c5db6f0ae8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:35.663797 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7281d-4a34-4b79-af8f-9c5db6f0ae8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:35.665854 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7281d-4a34-4b79-af8f-9c5db6f0ae8f service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:35.680103 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 879/1766] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:35.712531 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5e1dc-40b5-4cfe-8ab6-377e48526bca None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:35.716982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ec2b0-6db5-43a0-b7b3-497acb822442 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:35.717672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ec2b0-6db5-43a0-b7b3-497acb822442 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:35.717948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ec2b0-6db5-43a0-b7b3-497acb822442 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:35.718233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ec2b0-6db5-43a0-b7b3-497acb822442 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:35.728432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ec2b0-6db5-43a0-b7b3-497acb822442 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:35.730998 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ec2b0-6db5-43a0-b7b3-497acb822442 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:35.741677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:35.744685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 888/1767] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:35 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:35.779423 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7281d-4a34-4b79-af8f-9c5db6f0ae8f service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:35.781524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-47e7281d-4a34-4b79-af8f-9c5db6f0ae8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:35.874499 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7281d-4a34-4b79-af8f-9c5db6f0ae8f service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:35.883267 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79691b-9d47-4c2b-bdaa-4ff5211d1fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:35.886289 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79691b-9d47-4c2b-bdaa-4ff5211d1fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:35.886289 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79691b-9d47-4c2b-bdaa-4ff5211d1fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:35.886289 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79691b-9d47-4c2b-bdaa-4ff5211d1fd4 admin admin] PATH_INFO: `/v3/projects/3c658ffe76a24bf78d42c6458a482ee6` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:35.887318 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79691b-9d47-4c2b-bdaa-4ff5211d1fd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c658ffe76a24bf78d42c6458a482ee6)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:35.889641 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79691b-9d47-4c2b-bdaa-4ff5211d1fd4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:35.940761 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:36.026500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0d79691b-9d47-4c2b-bdaa-4ff5211d1fd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c658ffe76a24bf78d42c6458a482ee6', 'request_id': 'req-0d79691b-9d47-4c2b-bdaa-4ff5211d1fd4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:36.032126 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0d79691b-9d47-4c2b-bdaa-4ff5211d1fd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c658ffe76a24bf78d42c6458a482ee6', 'request_id': 'req-0d79691b-9d47-4c2b-bdaa-4ff5211d1fd4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:36.038850 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0d79691b-9d47-4c2b-bdaa-4ff5211d1fd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c658ffe76a24bf78d42c6458a482ee6', 'request_id': 'req-0d79691b-9d47-4c2b-bdaa-4ff5211d1fd4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:36.163992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0d79691b-9d47-4c2b-bdaa-4ff5211d1fd4 admin admin] The token cache is being invalidate because project 3c658ffe76a24bf78d42c6458a482ee6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:36.164283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0d79691b-9d47-4c2b-bdaa-4ff5211d1fd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:36.164903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0d79691b-9d47-4c2b-bdaa-4ff5211d1fd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:36.166578 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 880/1768] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:35 2022] DELETE /identity/v3/projects/3c658ffe76a24bf78d42c6458a482ee6 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:36.170566 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ec2b0-6db5-43a0-b7b3-497acb822442 service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:36.186946 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ec2b0-6db5-43a0-b7b3-497acb822442 service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:36.254862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055939b-4781-48c6-a39b-e78202673f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:36.255748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055939b-4781-48c6-a39b-e78202673f8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:36.256027 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055939b-4781-48c6-a39b-e78202673f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:36.256301 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055939b-4781-48c6-a39b-e78202673f8f admin admin] PATH_INFO: `/v3/users/0f2fd299c7544351b6d5e938b1180706` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:36.258580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055939b-4781-48c6-a39b-e78202673f8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f2fd299c7544351b6d5e938b1180706)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:36.260603 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055939b-4781-48c6-a39b-e78202673f8f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:36.468817 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7055939b-4781-48c6-a39b-e78202673f8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f2fd299c7544351b6d5e938b1180706'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:36.469578 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7055939b-4781-48c6-a39b-e78202673f8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f2fd299c7544351b6d5e938b1180706'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:36.477347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7055939b-4781-48c6-a39b-e78202673f8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f2fd299c7544351b6d5e938b1180706'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:36.478138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7055939b-4781-48c6-a39b-e78202673f8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f2fd299c7544351b6d5e938b1180706'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:36.563418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7055939b-4781-48c6-a39b-e78202673f8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f2fd299c7544351b6d5e938b1180706'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:36.590150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 889/1769] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:36 2022] DELETE /identity/v3/users/0f2fd299c7544351b6d5e938b1180706 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:38:36.738164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79691b-9d47-4c2b-bdaa-4ff5211d1fd4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:36.740889 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:36.743669 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0d79691b-9d47-4c2b-bdaa-4ff5211d1fd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:36.758750 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79691b-9d47-4c2b-bdaa-4ff5211d1fd4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:36.886945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10dbfed-ce07-44c9-96f5-4ca429bbe9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:36.888525 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10dbfed-ce07-44c9-96f5-4ca429bbe9f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:36.889126 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10dbfed-ce07-44c9-96f5-4ca429bbe9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:36.889748 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10dbfed-ce07-44c9-96f5-4ca429bbe9f8 admin admin] PATH_INFO: `/v3/users/eb90c9b7550b4b17bf5d228582d2dcb1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:36.892077 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10dbfed-ce07-44c9-96f5-4ca429bbe9f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb90c9b7550b4b17bf5d228582d2dcb1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:36.894409 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10dbfed-ce07-44c9-96f5-4ca429bbe9f8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:36.998135 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:37.132777 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055939b-4781-48c6-a39b-e78202673f8f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:37.134291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7055939b-4781-48c6-a39b-e78202673f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:37.143142 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a10dbfed-ce07-44c9-96f5-4ca429bbe9f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb90c9b7550b4b17bf5d228582d2dcb1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:37.146252 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a10dbfed-ce07-44c9-96f5-4ca429bbe9f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb90c9b7550b4b17bf5d228582d2dcb1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:37.171409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055939b-4781-48c6-a39b-e78202673f8f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:37.183577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a10dbfed-ce07-44c9-96f5-4ca429bbe9f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb90c9b7550b4b17bf5d228582d2dcb1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:37.196706 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a10dbfed-ce07-44c9-96f5-4ca429bbe9f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb90c9b7550b4b17bf5d228582d2dcb1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:37.201574 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594adc4-5bd4-4158-89af-197512aabe8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:37.202776 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594adc4-5bd4-4158-89af-197512aabe8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:37.203407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594adc4-5bd4-4158-89af-197512aabe8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:37.203872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594adc4-5bd4-4158-89af-197512aabe8c admin admin] PATH_INFO: `/v3/projects/17e12cf13f1749baac36fd2ac82d326d` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:37.205414 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594adc4-5bd4-4158-89af-197512aabe8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=17e12cf13f1749baac36fd2ac82d326d)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:37.207941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594adc4-5bd4-4158-89af-197512aabe8c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:37.232449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a10dbfed-ce07-44c9-96f5-4ca429bbe9f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb90c9b7550b4b17bf5d228582d2dcb1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:37.301584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:37.354438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 881/1770] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:36 2022] DELETE /identity/v3/users/eb90c9b7550b4b17bf5d228582d2dcb1 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:37.356157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce98111-35ae-4aad-aa91-4cedd9e5ecff None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:37.361400 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce98111-35ae-4aad-aa91-4cedd9e5ecff None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:37.361830 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce98111-35ae-4aad-aa91-4cedd9e5ecff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:37.378139 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b594adc4-5bd4-4158-89af-197512aabe8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17e12cf13f1749baac36fd2ac82d326d', 'request_id': 'req-b594adc4-5bd4-4158-89af-197512aabe8c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:37.393298 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b594adc4-5bd4-4158-89af-197512aabe8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17e12cf13f1749baac36fd2ac82d326d', 'request_id': 'req-b594adc4-5bd4-4158-89af-197512aabe8c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:37.398647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5ce98111-35ae-4aad-aa91-4cedd9e5ecff None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:37.400350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5ce98111-35ae-4aad-aa91-4cedd9e5ecff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:37.485972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 882/1771] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:37.504304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b594adc4-5bd4-4158-89af-197512aabe8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17e12cf13f1749baac36fd2ac82d326d', 'request_id': 'req-b594adc4-5bd4-4158-89af-197512aabe8c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:37.505384 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce98111-35ae-4aad-aa91-4cedd9e5ecff None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:37.507752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5ce98111-35ae-4aad-aa91-4cedd9e5ecff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:37.518978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b594adc4-5bd4-4158-89af-197512aabe8c admin admin] The token cache is being invalidate because project 17e12cf13f1749baac36fd2ac82d326d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:37.519275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b594adc4-5bd4-4158-89af-197512aabe8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:37.522613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b594adc4-5bd4-4158-89af-197512aabe8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:37.527997 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 890/1772] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:37 2022] DELETE /identity/v3/projects/17e12cf13f1749baac36fd2ac82d326d => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:37.529521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72caf9b-318c-4a7f-9bc2-afbe8a1ce97b None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:37.529808 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72caf9b-318c-4a7f-9bc2-afbe8a1ce97b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:37.530085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72caf9b-318c-4a7f-9bc2-afbe8a1ce97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:37.542521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:37.559448 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a72caf9b-318c-4a7f-9bc2-afbe8a1ce97b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:37.560736 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a72caf9b-318c-4a7f-9bc2-afbe8a1ce97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:37.575356 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 891/1773] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:37.583297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa6d19-2670-41f4-ac01-00f6564e3c34 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:37.583704 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa6d19-2670-41f4-ac01-00f6564e3c34 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:37.584066 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa6d19-2670-41f4-ac01-00f6564e3c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:37.598533 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce98111-35ae-4aad-aa91-4cedd9e5ecff None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:37.607560 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ebaa6d19-2670-41f4-ac01-00f6564e3c34 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:37.608952 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ebaa6d19-2670-41f4-ac01-00f6564e3c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:37.609951 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7149add-ac8d-4a14-aba3-160f81793661 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:37.610634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7149add-ac8d-4a14-aba3-160f81793661 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:37.610916 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7149add-ac8d-4a14-aba3-160f81793661 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:37.611198 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7149add-ac8d-4a14-aba3-160f81793661 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:37.614646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f7149add-ac8d-4a14-aba3-160f81793661 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:37.617916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 892/1774] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:38:37.623022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa6d19-2670-41f4-ac01-00f6564e3c34 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:37.632352 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7149add-ac8d-4a14-aba3-160f81793661 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:37.632877 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa6d19-2670-41f4-ac01-00f6564e3c34 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:37.634586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7149add-ac8d-4a14-aba3-160f81793661 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:37.638277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320c18b-46c4-48d0-b3c4-5f9bb712029e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:37.639084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320c18b-46c4-48d0-b3c4-5f9bb712029e admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:37.639474 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320c18b-46c4-48d0-b3c4-5f9bb712029e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:37.639852 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320c18b-46c4-48d0-b3c4-5f9bb712029e admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:37.640887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0320c18b-46c4-48d0-b3c4-5f9bb712029e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:37.641430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0320c18b-46c4-48d0-b3c4-5f9bb712029e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:37.646247 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0320c18b-46c4-48d0-b3c4-5f9bb712029e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:37.654798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 883/1775] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:37.679187 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 893/1776] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:37.695509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7149add-ac8d-4a14-aba3-160f81793661 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:37.695509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7149add-ac8d-4a14-aba3-160f81793661 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:37.700861 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c46120-3a46-49b0-8fc6-5e80153d24eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:37.701856 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c46120-3a46-49b0-8fc6-5e80153d24eb admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:37.702064 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c46120-3a46-49b0-8fc6-5e80153d24eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:37.702348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c46120-3a46-49b0-8fc6-5e80153d24eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:37.703371 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c46120-3a46-49b0-8fc6-5e80153d24eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:37.705709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c46120-3a46-49b0-8fc6-5e80153d24eb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:37.818043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320c18b-46c4-48d0-b3c4-5f9bb712029e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:37.820491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0320c18b-46c4-48d0-b3c4-5f9bb712029e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:37.847731 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320c18b-46c4-48d0-b3c4-5f9bb712029e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:37.850160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 884/1777] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:37 2022] POST /identity/v3/projects => generated 389 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:37.855362 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c46120-3a46-49b0-8fc6-5e80153d24eb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:37.985878 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d170071-5e78-44b7-8fcb-3c51877916fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:37.988270 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d170071-5e78-44b7-8fcb-3c51877916fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:37.988419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d170071-5e78-44b7-8fcb-3c51877916fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:37.988588 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d170071-5e78-44b7-8fcb-3c51877916fd admin admin] PATH_INFO: `/v3/users/89473e52c78240e88665e4b1c487cd12` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:37.994768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d170071-5e78-44b7-8fcb-3c51877916fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=89473e52c78240e88665e4b1c487cd12)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:38.002091 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d170071-5e78-44b7-8fcb-3c51877916fd admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:38.021218 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c46120-3a46-49b0-8fc6-5e80153d24eb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:38.047024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ed732-be71-4ca0-a1f9-3ca8cd88dffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:38.047942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ed732-be71-4ca0-a1f9-3ca8cd88dffc admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:38.048222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ed732-be71-4ca0-a1f9-3ca8cd88dffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:38.048496 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ed732-be71-4ca0-a1f9-3ca8cd88dffc admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:38.049353 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ed732-be71-4ca0-a1f9-3ca8cd88dffc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:38.051603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ed732-be71-4ca0-a1f9-3ca8cd88dffc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:38.370074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 885/1778] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:37 2022] POST /identity/v3/users => generated 295 bytes in 512 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:38:38.371844 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ed732-be71-4ca0-a1f9-3ca8cd88dffc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:38.384211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ed732-be71-4ca0-a1f9-3ca8cd88dffc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:38.405326 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91abc6-aeae-48f2-93d0-1f91733ffea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:38.405326 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91abc6-aeae-48f2-93d0-1f91733ffea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:38.405326 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91abc6-aeae-48f2-93d0-1f91733ffea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:38.405326 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91abc6-aeae-48f2-93d0-1f91733ffea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:38.405775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91abc6-aeae-48f2-93d0-1f91733ffea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:38.405775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91abc6-aeae-48f2-93d0-1f91733ffea7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:38.425929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 886/1779] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:38:38.434078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91abc6-aeae-48f2-93d0-1f91733ffea7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:38.457750 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91abc6-aeae-48f2-93d0-1f91733ffea7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:38.463299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c73986-3cb7-4a62-a143-bed43b014dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:38.463998 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c73986-3cb7-4a62-a143-bed43b014dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:38.464282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c73986-3cb7-4a62-a143-bed43b014dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:38.464559 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c73986-3cb7-4a62-a143-bed43b014dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:38.465342 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c73986-3cb7-4a62-a143-bed43b014dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:38.467045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c73986-3cb7-4a62-a143-bed43b014dd1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:38.485695 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 887/1780] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:38.489420 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c73986-3cb7-4a62-a143-bed43b014dd1 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:38.513252 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c73986-3cb7-4a62-a143-bed43b014dd1 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:38.529981 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67bdea-ca8a-468d-82ad-379b8c223b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:38.538510 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67bdea-ca8a-468d-82ad-379b8c223b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:38.538510 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67bdea-ca8a-468d-82ad-379b8c223b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:38.538510 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67bdea-ca8a-468d-82ad-379b8c223b50 admin admin] PATH_INFO: `/v3/projects/c46bf6de96964159b25fb03379a6ea88/users/7c388db83b3a49f8ace516a1286251a2/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:38.554210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67bdea-ca8a-468d-82ad-379b8c223b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c46bf6de96964159b25fb03379a6ea88, user_id=7c388db83b3a49f8ace516a1286251a2, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:38.558257 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67bdea-ca8a-468d-82ad-379b8c223b50 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:39.491721 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2d170071-5e78-44b7-8fcb-3c51877916fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89473e52c78240e88665e4b1c487cd12'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:39.491721 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2d170071-5e78-44b7-8fcb-3c51877916fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89473e52c78240e88665e4b1c487cd12'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:39.498108 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2d170071-5e78-44b7-8fcb-3c51877916fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89473e52c78240e88665e4b1c487cd12'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:39.499164 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2d170071-5e78-44b7-8fcb-3c51877916fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89473e52c78240e88665e4b1c487cd12'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:39.512862 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:39.546075 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 888/1781] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:38 2022] PUT /identity/v3/projects/c46bf6de96964159b25fb03379a6ea88/users/7c388db83b3a49f8ace516a1286251a2/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 1043 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:39.547320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67bdea-ca8a-468d-82ad-379b8c223b50 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:39.550149 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-df67bdea-ca8a-468d-82ad-379b8c223b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:39.677923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67bdea-ca8a-468d-82ad-379b8c223b50 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:39.701865 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec97d0-e7cf-487f-b9ce-b781224e0e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:39.702869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec97d0-e7cf-487f-b9ce-b781224e0e2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:39.703080 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec97d0-e7cf-487f-b9ce-b781224e0e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:39.703364 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec97d0-e7cf-487f-b9ce-b781224e0e2f admin admin] PATH_INFO: `/v3/projects/017d2b198d154b2181ea4da8a8d9ea54` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:39.704587 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec97d0-e7cf-487f-b9ce-b781224e0e2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=017d2b198d154b2181ea4da8a8d9ea54)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:39.706728 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec97d0-e7cf-487f-b9ce-b781224e0e2f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:39.718303 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2d170071-5e78-44b7-8fcb-3c51877916fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89473e52c78240e88665e4b1c487cd12'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:39.758884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 894/1782] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:37 2022] DELETE /identity/v3/users/89473e52c78240e88665e4b1c487cd12 => generated 0 bytes in 1941 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:38:39.760535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021e30f-f511-4d3a-9adb-5e33cb65799a None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:39.760985 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021e30f-f511-4d3a-9adb-5e33cb65799a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:39.764159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021e30f-f511-4d3a-9adb-5e33cb65799a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:39.804451 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-4021e30f-f511-4d3a-9adb-5e33cb65799a None None] MFA Rules not processed for user `7c388db83b3a49f8ace516a1286251a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:39.806610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4021e30f-f511-4d3a-9adb-5e33cb65799a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:39.856256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-94ec97d0-e7cf-487f-b9ce-b781224e0e2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '017d2b198d154b2181ea4da8a8d9ea54', 'request_id': 'req-94ec97d0-e7cf-487f-b9ce-b781224e0e2f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:39.865882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-94ec97d0-e7cf-487f-b9ce-b781224e0e2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '017d2b198d154b2181ea4da8a8d9ea54', 'request_id': 'req-94ec97d0-e7cf-487f-b9ce-b781224e0e2f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:39.873474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-94ec97d0-e7cf-487f-b9ce-b781224e0e2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '017d2b198d154b2181ea4da8a8d9ea54', 'request_id': 'req-94ec97d0-e7cf-487f-b9ce-b781224e0e2f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:39.900345 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 895/1783] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:39.900345 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34284396-97ee-4166-b812-75b0ebd00c2c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:39.900345 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34284396-97ee-4166-b812-75b0ebd00c2c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:39.900345 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34284396-97ee-4166-b812-75b0ebd00c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:39.913126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-34284396-97ee-4166-b812-75b0ebd00c2c None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:39.914338 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-34284396-97ee-4166-b812-75b0ebd00c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:39.987958 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 896/1784] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:39.988316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34284396-97ee-4166-b812-75b0ebd00c2c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:39.991468 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-34284396-97ee-4166-b812-75b0ebd00c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:40.006128 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34284396-97ee-4166-b812-75b0ebd00c2c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:40.011587 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ad488-8832-42e1-a9c7-8db93fa32873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:40.012258 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ad488-8832-42e1-a9c7-8db93fa32873 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:40.012543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ad488-8832-42e1-a9c7-8db93fa32873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:40.012828 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ad488-8832-42e1-a9c7-8db93fa32873 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:40.014022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ad488-8832-42e1-a9c7-8db93fa32873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:40.021658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-94ec97d0-e7cf-487f-b9ce-b781224e0e2f admin admin] The token cache is being invalidate because project 017d2b198d154b2181ea4da8a8d9ea54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:40.025282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-94ec97d0-e7cf-487f-b9ce-b781224e0e2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:40.025282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-94ec97d0-e7cf-487f-b9ce-b781224e0e2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:40.025469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ad488-8832-42e1-a9c7-8db93fa32873 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:40.025551 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 889/1785] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:39 2022] DELETE /identity/v3/projects/017d2b198d154b2181ea4da8a8d9ea54 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:40.025919 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec97d0-e7cf-487f-b9ce-b781224e0e2f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:40.027334 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-94ec97d0-e7cf-487f-b9ce-b781224e0e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:40.097517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec97d0-e7cf-487f-b9ce-b781224e0e2f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:40.102261 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1b98c-5009-4e09-84b2-940d3c908e2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:40.102925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1b98c-5009-4e09-84b2-940d3c908e2c service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:40.103409 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1b98c-5009-4e09-84b2-940d3c908e2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:40.103511 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1b98c-5009-4e09-84b2-940d3c908e2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:40.105387 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ecc1b98c-5009-4e09-84b2-940d3c908e2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:40.118991 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1b98c-5009-4e09-84b2-940d3c908e2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:40.120821 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1b98c-5009-4e09-84b2-940d3c908e2c service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:40.131230 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 890/1786] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:40.132650 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1b98c-5009-4e09-84b2-940d3c908e2c service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:40.134050 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ecc1b98c-5009-4e09-84b2-940d3c908e2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:40.146282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1b98c-5009-4e09-84b2-940d3c908e2c service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:40.156591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96582c-e405-4e05-8ac4-f061637c4e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:40.161087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 897/1787] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:39 2022] POST /identity/v3/projects => generated 389 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:40.161388 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96582c-e405-4e05-8ac4-f061637c4e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:40.161388 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96582c-e405-4e05-8ac4-f061637c4e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:40.161388 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96582c-e405-4e05-8ac4-f061637c4e73 admin admin] PATH_INFO: `/v3/users/9d296aa5db0c4dc5a2799bd9bf64440e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:40.161388 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96582c-e405-4e05-8ac4-f061637c4e73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d296aa5db0c4dc5a2799bd9bf64440e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:40.162139 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ad488-8832-42e1-a9c7-8db93fa32873 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:40.162257 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96582c-e405-4e05-8ac4-f061637c4e73 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:40.164088 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2e1ad488-8832-42e1-a9c7-8db93fa32873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:40.246651 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ad488-8832-42e1-a9c7-8db93fa32873 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:40.253174 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323f4f62-3e94-4056-94e6-c97f6f0e0461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:40.254522 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f4f62-3e94-4056-94e6-c97f6f0e0461 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:40.254852 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f4f62-3e94-4056-94e6-c97f6f0e0461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:40.255364 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f4f62-3e94-4056-94e6-c97f6f0e0461 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:40.257204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323f4f62-3e94-4056-94e6-c97f6f0e0461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:40.261523 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323f4f62-3e94-4056-94e6-c97f6f0e0461 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:40.392578 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 898/1788] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:40 2022] POST /identity/v3/users => generated 301 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:40.397549 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323f4f62-3e94-4056-94e6-c97f6f0e0461 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:40.417993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323f4f62-3e94-4056-94e6-c97f6f0e0461 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:40.421408 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6d96582c-e405-4e05-8ac4-f061637c4e73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d296aa5db0c4dc5a2799bd9bf64440e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:40.422537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6d96582c-e405-4e05-8ac4-f061637c4e73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d296aa5db0c4dc5a2799bd9bf64440e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:40.433503 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee43ec-44c7-4f14-9554-be135a9897d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:40.434252 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee43ec-44c7-4f14-9554-be135a9897d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:40.434535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee43ec-44c7-4f14-9554-be135a9897d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:40.434794 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee43ec-44c7-4f14-9554-be135a9897d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:40.435538 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee43ec-44c7-4f14-9554-be135a9897d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:40.445765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee43ec-44c7-4f14-9554-be135a9897d0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:40.445863 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6d96582c-e405-4e05-8ac4-f061637c4e73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d296aa5db0c4dc5a2799bd9bf64440e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:40.445863 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6d96582c-e405-4e05-8ac4-f061637c4e73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d296aa5db0c4dc5a2799bd9bf64440e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:40.459472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 899/1789] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:40.460473 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee43ec-44c7-4f14-9554-be135a9897d0 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:40.462961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6d96582c-e405-4e05-8ac4-f061637c4e73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d296aa5db0c4dc5a2799bd9bf64440e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:40.468106 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6dee43ec-44c7-4f14-9554-be135a9897d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:40.482979 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee43ec-44c7-4f14-9554-be135a9897d0 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:40.488953 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f7d7c-9271-48a4-bc0b-b1d23a6ad6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:40.489985 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f7d7c-9271-48a4-bc0b-b1d23a6ad6b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:40.490340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f7d7c-9271-48a4-bc0b-b1d23a6ad6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:40.490708 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f7d7c-9271-48a4-bc0b-b1d23a6ad6b5 admin admin] PATH_INFO: `/v3/projects/c74cd0d8bd3945fb9fbc118ca018f951/users/2a65035b21a8486ab8878b0d1269be33/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:40.514693 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f7d7c-9271-48a4-bc0b-b1d23a6ad6b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c74cd0d8bd3945fb9fbc118ca018f951, user_id=2a65035b21a8486ab8878b0d1269be33, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:40.515319 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 891/1790] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:40 2022] DELETE /identity/v3/users/9d296aa5db0c4dc5a2799bd9bf64440e => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:40.518524 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f7d7c-9271-48a4-bc0b-b1d23a6ad6b5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:40.589394 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:40.618457 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 900/1791] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:40 2022] PUT /identity/v3/projects/c74cd0d8bd3945fb9fbc118ca018f951/users/2a65035b21a8486ab8878b0d1269be33/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:40.628758 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96582c-e405-4e05-8ac4-f061637c4e73 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:40.790745 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96582c-e405-4e05-8ac4-f061637c4e73 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:40.912294 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dedc6d-c502-486e-859d-a13df4a042c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:40.913716 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dedc6d-c502-486e-859d-a13df4a042c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:40.914516 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dedc6d-c502-486e-859d-a13df4a042c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:40.914516 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dedc6d-c502-486e-859d-a13df4a042c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:40.915058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dedc6d-c502-486e-859d-a13df4a042c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:40.918132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dedc6d-c502-486e-859d-a13df4a042c0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:40.940581 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 892/1792] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:38:40.940887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f7d7c-9271-48a4-bc0b-b1d23a6ad6b5 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:40.965051 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f7d7c-9271-48a4-bc0b-b1d23a6ad6b5 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:40.965051 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0396bf0-eb1f-4f35-826e-941ce9069f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:40.965051 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0396bf0-eb1f-4f35-826e-941ce9069f08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:40.965051 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0396bf0-eb1f-4f35-826e-941ce9069f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:40.966512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0396bf0-eb1f-4f35-826e-941ce9069f08 admin admin] PATH_INFO: `/v3/projects/c74cd0d8bd3945fb9fbc118ca018f951/users/2a65035b21a8486ab8878b0d1269be33/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:40.968797 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0396bf0-eb1f-4f35-826e-941ce9069f08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c74cd0d8bd3945fb9fbc118ca018f951, user_id=2a65035b21a8486ab8878b0d1269be33, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:40.972806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0396bf0-eb1f-4f35-826e-941ce9069f08 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:41.005212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d37e6b-2df3-4310-8f19-1d55c2c11fe3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:41.005504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d37e6b-2df3-4310-8f19-1d55c2c11fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:41.005783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d37e6b-2df3-4310-8f19-1d55c2c11fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:41.007382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 901/1793] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:40 2022] PUT /identity/v3/projects/c74cd0d8bd3945fb9fbc118ca018f951/users/2a65035b21a8486ab8878b0d1269be33/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:38:41.062537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-04d37e6b-2df3-4310-8f19-1d55c2c11fe3 None None] MFA Rules not processed for user `2a65035b21a8486ab8878b0d1269be33`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:41.063838 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-04d37e6b-2df3-4310-8f19-1d55c2c11fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:41.168625 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0396bf0-eb1f-4f35-826e-941ce9069f08 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:41.175123 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b0396bf0-eb1f-4f35-826e-941ce9069f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:41.198025 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:41.237713 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 893/1794] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:41.286454 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d37e6b-2df3-4310-8f19-1d55c2c11fe3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:41.289103 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-04d37e6b-2df3-4310-8f19-1d55c2c11fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:41.304945 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0396bf0-eb1f-4f35-826e-941ce9069f08 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:41.319072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470cc55-7089-4dbd-8596-2cecb4440948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:41.319739 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470cc55-7089-4dbd-8596-2cecb4440948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:41.320041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470cc55-7089-4dbd-8596-2cecb4440948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:41.320742 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470cc55-7089-4dbd-8596-2cecb4440948 admin admin] PATH_INFO: `/v3/projects/fb4a6b81990b40ac958b028bff582296` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:41.322351 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470cc55-7089-4dbd-8596-2cecb4440948 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb4a6b81990b40ac958b028bff582296)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:41.325097 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470cc55-7089-4dbd-8596-2cecb4440948 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:41.391030 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d37e6b-2df3-4310-8f19-1d55c2c11fe3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:41.398236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ac93f-d403-4fc4-9f6b-6e447009264c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:41.399091 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ac93f-d403-4fc4-9f6b-6e447009264c service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:41.399376 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ac93f-d403-4fc4-9f6b-6e447009264c service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:41.399651 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ac93f-d403-4fc4-9f6b-6e447009264c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:41.410222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ac93f-d403-4fc4-9f6b-6e447009264c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:41.415716 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ac93f-d403-4fc4-9f6b-6e447009264c service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:41.433080 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 894/1795] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:38:41.499407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d470cc55-7089-4dbd-8596-2cecb4440948 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb4a6b81990b40ac958b028bff582296', 'request_id': 'req-d470cc55-7089-4dbd-8596-2cecb4440948'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:41.510010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d470cc55-7089-4dbd-8596-2cecb4440948 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb4a6b81990b40ac958b028bff582296', 'request_id': 'req-d470cc55-7089-4dbd-8596-2cecb4440948'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:41.549028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ac93f-d403-4fc4-9f6b-6e447009264c service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:41.549857 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b56ac93f-d403-4fc4-9f6b-6e447009264c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:41.576381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ac93f-d403-4fc4-9f6b-6e447009264c service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:41.617409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d470cc55-7089-4dbd-8596-2cecb4440948 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb4a6b81990b40ac958b028bff582296', 'request_id': 'req-d470cc55-7089-4dbd-8596-2cecb4440948'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:41.649756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d470cc55-7089-4dbd-8596-2cecb4440948 admin admin] The token cache is being invalidate because project fb4a6b81990b40ac958b028bff582296 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:41.650565 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d470cc55-7089-4dbd-8596-2cecb4440948 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:41.651916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d470cc55-7089-4dbd-8596-2cecb4440948 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:41.654762 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 902/1796] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:41 2022] DELETE /identity/v3/projects/fb4a6b81990b40ac958b028bff582296 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:41.656706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470cc55-7089-4dbd-8596-2cecb4440948 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:41.660584 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d470cc55-7089-4dbd-8596-2cecb4440948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:41.672729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dfd53c-9b60-4ae3-ac7c-714c51e022f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:41.676906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfd53c-9b60-4ae3-ac7c-714c51e022f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:41.676906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfd53c-9b60-4ae3-ac7c-714c51e022f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:41.676906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfd53c-9b60-4ae3-ac7c-714c51e022f6 admin admin] PATH_INFO: `/v3/projects/5128fff40ebd431fb51cff07772b17bb` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:41.676906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dfd53c-9b60-4ae3-ac7c-714c51e022f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5128fff40ebd431fb51cff07772b17bb)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:41.677911 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dfd53c-9b60-4ae3-ac7c-714c51e022f6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:41.741841 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470cc55-7089-4dbd-8596-2cecb4440948 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:41.751502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a1fcf-03a5-421e-ac00-a35749a90ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:41.752433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a1fcf-03a5-421e-ac00-a35749a90ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:41.752709 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a1fcf-03a5-421e-ac00-a35749a90ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:41.752983 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a1fcf-03a5-421e-ac00-a35749a90ac1 admin admin] PATH_INFO: `/v3/users/7c388db83b3a49f8ace516a1286251a2` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:41.754249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a1fcf-03a5-421e-ac00-a35749a90ac1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c388db83b3a49f8ace516a1286251a2)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:41.756205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a1fcf-03a5-421e-ac00-a35749a90ac1 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:41.783159 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c3dfd53c-9b60-4ae3-ac7c-714c51e022f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5128fff40ebd431fb51cff07772b17bb', 'request_id': 'req-c3dfd53c-9b60-4ae3-ac7c-714c51e022f6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:41.790004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c3dfd53c-9b60-4ae3-ac7c-714c51e022f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5128fff40ebd431fb51cff07772b17bb', 'request_id': 'req-c3dfd53c-9b60-4ae3-ac7c-714c51e022f6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:41.796563 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c3dfd53c-9b60-4ae3-ac7c-714c51e022f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5128fff40ebd431fb51cff07772b17bb', 'request_id': 'req-c3dfd53c-9b60-4ae3-ac7c-714c51e022f6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:41.913273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c3dfd53c-9b60-4ae3-ac7c-714c51e022f6 admin admin] The token cache is being invalidate because project 5128fff40ebd431fb51cff07772b17bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:41.913831 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c3dfd53c-9b60-4ae3-ac7c-714c51e022f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:41.914923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c3dfd53c-9b60-4ae3-ac7c-714c51e022f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:41.916661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 895/1797] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:41 2022] DELETE /identity/v3/projects/5128fff40ebd431fb51cff07772b17bb => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:38:41.918273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b79b92-4478-4e43-8258-258c6f642f49 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:41.918855 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b79b92-4478-4e43-8258-258c6f642f49 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:41.919199 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b79b92-4478-4e43-8258-258c6f642f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:41.940363 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-67b79b92-4478-4e43-8258-258c6f642f49 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:41.941719 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-67b79b92-4478-4e43-8258-258c6f642f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:41.954798 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-698a1fcf-03a5-421e-ac00-a35749a90ac1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c388db83b3a49f8ace516a1286251a2'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:41.956037 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-698a1fcf-03a5-421e-ac00-a35749a90ac1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c388db83b3a49f8ace516a1286251a2'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:41.965304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-698a1fcf-03a5-421e-ac00-a35749a90ac1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c388db83b3a49f8ace516a1286251a2'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:41.966559 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-698a1fcf-03a5-421e-ac00-a35749a90ac1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c388db83b3a49f8ace516a1286251a2'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:42.008740 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 896/1798] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:42.009722 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b79b92-4478-4e43-8258-258c6f642f49 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:42.011204 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-67b79b92-4478-4e43-8258-258c6f642f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:42.026256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b79b92-4478-4e43-8258-258c6f642f49 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:42.032315 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676008f-4b7b-45c3-88a2-88c692defdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:42.032315 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676008f-4b7b-45c3-88a2-88c692defdb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:42.032716 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676008f-4b7b-45c3-88a2-88c692defdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:42.033191 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676008f-4b7b-45c3-88a2-88c692defdb5 admin admin] PATH_INFO: `/v3/users/49d3620d0d3c4924ba3e15b0974bc357` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:42.034967 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676008f-4b7b-45c3-88a2-88c692defdb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49d3620d0d3c4924ba3e15b0974bc357)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:42.037492 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676008f-4b7b-45c3-88a2-88c692defdb5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:42.049041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-698a1fcf-03a5-421e-ac00-a35749a90ac1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c388db83b3a49f8ace516a1286251a2'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:42.085788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 903/1799] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:41 2022] DELETE /identity/v3/users/7c388db83b3a49f8ace516a1286251a2 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:42.089118 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171ba5c-fe0e-46ae-a503-abc07957279f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:42.089340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171ba5c-fe0e-46ae-a503-abc07957279f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:42.089618 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171ba5c-fe0e-46ae-a503-abc07957279f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:42.135231 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-6171ba5c-fe0e-46ae-a503-abc07957279f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:42.136603 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6171ba5c-fe0e-46ae-a503-abc07957279f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:42.233763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 904/1800] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:42.236525 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a3c63-29ab-4463-91d0-f242503fc9d9 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:42.236885 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a3c63-29ab-4463-91d0-f242503fc9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:42.237250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a3c63-29ab-4463-91d0-f242503fc9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:42.271055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-391a3c63-29ab-4463-91d0-f242503fc9d9 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:42.275707 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-391a3c63-29ab-4463-91d0-f242503fc9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:42.281523 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3676008f-4b7b-45c3-88a2-88c692defdb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49d3620d0d3c4924ba3e15b0974bc357'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:42.282255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3676008f-4b7b-45c3-88a2-88c692defdb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49d3620d0d3c4924ba3e15b0974bc357'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:42.285835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 905/1801] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:42.288104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad34b3d-fa93-496c-bc80-7cf6b71d3a59 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:42.289032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad34b3d-fa93-496c-bc80-7cf6b71d3a59 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:42.289032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad34b3d-fa93-496c-bc80-7cf6b71d3a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:42.315373 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-dad34b3d-fa93-496c-bc80-7cf6b71d3a59 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:42.316808 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-dad34b3d-fa93-496c-bc80-7cf6b71d3a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:42.318447 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3676008f-4b7b-45c3-88a2-88c692defdb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49d3620d0d3c4924ba3e15b0974bc357'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:42.321513 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3676008f-4b7b-45c3-88a2-88c692defdb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49d3620d0d3c4924ba3e15b0974bc357'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:42.340639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 906/1802] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:42.340639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381eb8ba-2c5b-452f-ac63-c47ff9c13d84 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:42.340639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381eb8ba-2c5b-452f-ac63-c47ff9c13d84 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:42.340639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381eb8ba-2c5b-452f-ac63-c47ff9c13d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:42.360813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3676008f-4b7b-45c3-88a2-88c692defdb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49d3620d0d3c4924ba3e15b0974bc357'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:42.379705 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-381eb8ba-2c5b-452f-ac63-c47ff9c13d84 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:42.381869 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-381eb8ba-2c5b-452f-ac63-c47ff9c13d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:42.396636 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 907/1803] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:42.396636 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381eb8ba-2c5b-452f-ac63-c47ff9c13d84 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:42.399104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-381eb8ba-2c5b-452f-ac63-c47ff9c13d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:42.414862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381eb8ba-2c5b-452f-ac63-c47ff9c13d84 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:42.429624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f382de-ade6-4655-925d-ab45511f751c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:42.429624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f382de-ade6-4655-925d-ab45511f751c admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:42.429624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f382de-ade6-4655-925d-ab45511f751c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:42.429624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f382de-ade6-4655-925d-ab45511f751c admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:42.430206 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f382de-ade6-4655-925d-ab45511f751c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:42.431147 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f382de-ade6-4655-925d-ab45511f751c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:42.433106 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f382de-ade6-4655-925d-ab45511f751c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:42.450799 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 908/1804] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:42.452506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f382de-ade6-4655-925d-ab45511f751c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:42.470352 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 897/1805] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:42 2022] DELETE /identity/v3/users/49d3620d0d3c4924ba3e15b0974bc357 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:42.470352 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676008f-4b7b-45c3-88a2-88c692defdb5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:42.470754 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f382de-ade6-4655-925d-ab45511f751c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:42.483975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3676008f-4b7b-45c3-88a2-88c692defdb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:42.488455 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:42.571750 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ff975-fc9f-44ff-a3e3-de368a513b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:42.574027 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ff975-fc9f-44ff-a3e3-de368a513b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:42.578717 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ff975-fc9f-44ff-a3e3-de368a513b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:42.578717 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ff975-fc9f-44ff-a3e3-de368a513b43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:42.578717 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ff975-fc9f-44ff-a3e3-de368a513b43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:42.578717 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ff975-fc9f-44ff-a3e3-de368a513b43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:42.578717 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ff975-fc9f-44ff-a3e3-de368a513b43 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:42.611900 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 909/1806] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:42.612736 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ff975-fc9f-44ff-a3e3-de368a513b43 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:42.621615 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676008f-4b7b-45c3-88a2-88c692defdb5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:42.626113 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae360959-329d-4771-9232-1405ac837855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:42.626744 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae360959-329d-4771-9232-1405ac837855 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:42.627110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae360959-329d-4771-9232-1405ac837855 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:42.627392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae360959-329d-4771-9232-1405ac837855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:42.627608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ff975-fc9f-44ff-a3e3-de368a513b43 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:42.646559 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae360959-329d-4771-9232-1405ac837855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:42.646559 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae360959-329d-4771-9232-1405ac837855 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:42.646891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89df52-1190-475c-8208-75c5ba346c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:42.646891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89df52-1190-475c-8208-75c5ba346c42 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:42.646891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89df52-1190-475c-8208-75c5ba346c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:42.646891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89df52-1190-475c-8208-75c5ba346c42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:42.647756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89df52-1190-475c-8208-75c5ba346c42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:42.648766 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89df52-1190-475c-8208-75c5ba346c42 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:42.651588 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 898/1807] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:42.654667 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077924b-2059-4f73-b65b-ab263b313d6c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:42.654968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077924b-2059-4f73-b65b-ab263b313d6c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:42.655295 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077924b-2059-4f73-b65b-ab263b313d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:42.676969 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b077924b-2059-4f73-b65b-ab263b313d6c None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:42.678265 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b077924b-2059-4f73-b65b-ab263b313d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:42.690290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 899/1808] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:42.696935 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077924b-2059-4f73-b65b-ab263b313d6c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:42.702131 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077924b-2059-4f73-b65b-ab263b313d6c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:42.710938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae40f6-f10e-45d8-a77f-26c8bd5ce373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:42.724449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae40f6-f10e-45d8-a77f-26c8bd5ce373 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:42.724449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae40f6-f10e-45d8-a77f-26c8bd5ce373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:42.724449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae40f6-f10e-45d8-a77f-26c8bd5ce373 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:42.724449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae40f6-f10e-45d8-a77f-26c8bd5ce373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:42.724449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae40f6-f10e-45d8-a77f-26c8bd5ce373 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:42.764070 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 910/1809] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:42 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:42.764070 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89df52-1190-475c-8208-75c5ba346c42 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:42.767662 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1c89df52-1190-475c-8208-75c5ba346c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:42.843189 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:42.843877 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 900/1810] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:42 2022] POST /identity/v3/projects => generated 365 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:42.852373 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae40f6-f10e-45d8-a77f-26c8bd5ce373 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:42.873982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89df52-1190-475c-8208-75c5ba346c42 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:42.951685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcc6d0-2485-4b4e-b26e-822b9b046dcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:42.952193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcc6d0-2485-4b4e-b26e-822b9b046dcc service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:42.952467 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcc6d0-2485-4b4e-b26e-822b9b046dcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:42.952761 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcc6d0-2485-4b4e-b26e-822b9b046dcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:42.973339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae40f6-f10e-45d8-a77f-26c8bd5ce373 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:42.978533 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ddd161-3c41-4d7d-8dee-098623983677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:42.979224 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddd161-3c41-4d7d-8dee-098623983677 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:42.979529 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddd161-3c41-4d7d-8dee-098623983677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:42.979777 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddd161-3c41-4d7d-8dee-098623983677 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:42.980478 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ddd161-3c41-4d7d-8dee-098623983677 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:42.982485 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ddd161-3c41-4d7d-8dee-098623983677 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:43.045693 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fcc6d0-2485-4b4e-b26e-822b9b046dcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:43.047221 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fcc6d0-2485-4b4e-b26e-822b9b046dcc service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:43.057799 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 911/1811] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:43.062025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcc6d0-2485-4b4e-b26e-822b9b046dcc service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:43.076883 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcc6d0-2485-4b4e-b26e-822b9b046dcc service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:43.087412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b93f5-d4c0-4e2a-af17-1ec0835c5e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:43.088110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b93f5-d4c0-4e2a-af17-1ec0835c5e4a admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:43.088388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b93f5-d4c0-4e2a-af17-1ec0835c5e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:43.088684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b93f5-d4c0-4e2a-af17-1ec0835c5e4a admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:43.090545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b93f5-d4c0-4e2a-af17-1ec0835c5e4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:43.092449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b93f5-d4c0-4e2a-af17-1ec0835c5e4a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:43.114606 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 901/1812] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:42 2022] POST /identity/v3/users => generated 288 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:43.115593 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ddd161-3c41-4d7d-8dee-098623983677 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:43.120211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f9ddd161-3c41-4d7d-8dee-098623983677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:43.137879 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ddd161-3c41-4d7d-8dee-098623983677 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:43.143782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c8bec-e8c4-43cd-9943-f8ebd1f24c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:43.144865 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c8bec-e8c4-43cd-9943-f8ebd1f24c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:43.145746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c8bec-e8c4-43cd-9943-f8ebd1f24c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:43.146033 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c8bec-e8c4-43cd-9943-f8ebd1f24c41 admin admin] PATH_INFO: `/v3/users/2cd827b8e9f34dae8bdafdf3f8f9ba07` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:43.148009 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c8bec-e8c4-43cd-9943-f8ebd1f24c41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cd827b8e9f34dae8bdafdf3f8f9ba07)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:43.150382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c8bec-e8c4-43cd-9943-f8ebd1f24c41 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:43.177049 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 912/1813] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:43 2022] POST /identity/v3/users => generated 283 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:43.177049 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b93f5-d4c0-4e2a-af17-1ec0835c5e4a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:43.200763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b93f5-d4c0-4e2a-af17-1ec0835c5e4a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:43.213483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9aeec-c01d-48b8-835d-4c4d32595ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:43.213483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9aeec-c01d-48b8-835d-4c4d32595ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:43.213483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9aeec-c01d-48b8-835d-4c4d32595ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:43.214065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9aeec-c01d-48b8-835d-4c4d32595ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:43.217766 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9aeec-c01d-48b8-835d-4c4d32595ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:43.223162 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9aeec-c01d-48b8-835d-4c4d32595ba9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:43.243960 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 913/1814] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:43.245396 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9aeec-c01d-48b8-835d-4c4d32595ba9 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:43.260283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9aeec-c01d-48b8-835d-4c4d32595ba9 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:43.266010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b917134-e59f-40b4-a34e-8354cc57ee11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:43.266709 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b917134-e59f-40b4-a34e-8354cc57ee11 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:43.268100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b917134-e59f-40b4-a34e-8354cc57ee11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:43.268100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b917134-e59f-40b4-a34e-8354cc57ee11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:43.268100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b917134-e59f-40b4-a34e-8354cc57ee11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:43.269689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b917134-e59f-40b4-a34e-8354cc57ee11 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:43.281326 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 914/1815] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:43.281988 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b917134-e59f-40b4-a34e-8354cc57ee11 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:43.299233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b917134-e59f-40b4-a34e-8354cc57ee11 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:43.312034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a8b11-76fe-4544-9b44-48c97722859d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:43.312335 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a8b11-76fe-4544-9b44-48c97722859d admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:43.313034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a8b11-76fe-4544-9b44-48c97722859d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:43.313034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a8b11-76fe-4544-9b44-48c97722859d admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:43.313740 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a8b11-76fe-4544-9b44-48c97722859d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:43.315620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a8b11-76fe-4544-9b44-48c97722859d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:43.327297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 915/1816] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:43.330787 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a8b11-76fe-4544-9b44-48c97722859d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:43.356420 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a8b11-76fe-4544-9b44-48c97722859d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:43.362180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc5e66-adb0-4742-a262-c33cf65d8abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:43.366358 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc5e66-adb0-4742-a262-c33cf65d8abd admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:43.366358 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc5e66-adb0-4742-a262-c33cf65d8abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:43.366358 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc5e66-adb0-4742-a262-c33cf65d8abd admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:43.366358 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc5e66-adb0-4742-a262-c33cf65d8abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:43.366700 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc5e66-adb0-4742-a262-c33cf65d8abd admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:43.386216 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 916/1817] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:43.389213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc5e66-adb0-4742-a262-c33cf65d8abd admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:43.407325 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc5e66-adb0-4742-a262-c33cf65d8abd admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:43.411690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-790c8bec-e8c4-43cd-9943-f8ebd1f24c41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cd827b8e9f34dae8bdafdf3f8f9ba07'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:43.413640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-790c8bec-e8c4-43cd-9943-f8ebd1f24c41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cd827b8e9f34dae8bdafdf3f8f9ba07'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:43.417367 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3e13e-d0b2-4fc2-99d9-01a163f59e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:43.423374 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3e13e-d0b2-4fc2-99d9-01a163f59e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:43.423374 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3e13e-d0b2-4fc2-99d9-01a163f59e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:43.423374 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3e13e-d0b2-4fc2-99d9-01a163f59e48 admin admin] PATH_INFO: `/v3/projects/1921d0de571f454eb510fbee6aa30eb7/users/059d623a01ea48d79fb6fbebdb92551f/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:43.429678 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-790c8bec-e8c4-43cd-9943-f8ebd1f24c41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cd827b8e9f34dae8bdafdf3f8f9ba07'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:43.434806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-790c8bec-e8c4-43cd-9943-f8ebd1f24c41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cd827b8e9f34dae8bdafdf3f8f9ba07'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:43.446901 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:43.459138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb3e13e-d0b2-4fc2-99d9-01a163f59e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1921d0de571f454eb510fbee6aa30eb7, user_id=059d623a01ea48d79fb6fbebdb92551f, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:43.461789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-790c8bec-e8c4-43cd-9943-f8ebd1f24c41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cd827b8e9f34dae8bdafdf3f8f9ba07'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:43.464505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb3e13e-d0b2-4fc2-99d9-01a163f59e48 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:43.564859 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 902/1818] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:43 2022] DELETE /identity/v3/users/2cd827b8e9f34dae8bdafdf3f8f9ba07 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:43.565975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c8bec-e8c4-43cd-9943-f8ebd1f24c41 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:43.567904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-790c8bec-e8c4-43cd-9943-f8ebd1f24c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:43.599443 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 917/1819] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:43 2022] PUT /identity/v3/projects/1921d0de571f454eb510fbee6aa30eb7/users/059d623a01ea48d79fb6fbebdb92551f/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:43.601931 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3e13e-d0b2-4fc2-99d9-01a163f59e48 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:43.611540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1fb3e13e-d0b2-4fc2-99d9-01a163f59e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:43.719490 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c8bec-e8c4-43cd-9943-f8ebd1f24c41 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:43.720005 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3e13e-d0b2-4fc2-99d9-01a163f59e48 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:43.724606 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3f4e6-609a-49b5-8704-24946a3cb3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:43.726223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3f4e6-609a-49b5-8704-24946a3cb3ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:43.726671 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3f4e6-609a-49b5-8704-24946a3cb3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:43.727599 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3f4e6-609a-49b5-8704-24946a3cb3ef admin admin] PATH_INFO: `/v3/projects/c46bf6de96964159b25fb03379a6ea88` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:43.728392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3f4e6-609a-49b5-8704-24946a3cb3ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=c46bf6de96964159b25fb03379a6ea88)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:43.730518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3f4e6-609a-49b5-8704-24946a3cb3ef admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:43.739841 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64132fc9-85b9-45ed-9fd0-e6c3d6284ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:43.745548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64132fc9-85b9-45ed-9fd0-e6c3d6284ba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:43.749044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64132fc9-85b9-45ed-9fd0-e6c3d6284ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:43.749649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64132fc9-85b9-45ed-9fd0-e6c3d6284ba2 admin admin] PATH_INFO: `/v3/projects/fa5d6717df8147e88a9afaea2a5d1a30/users/00a810e782154d838e5b73368f3db35e/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:43.770491 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:43.789857 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64132fc9-85b9-45ed-9fd0-e6c3d6284ba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa5d6717df8147e88a9afaea2a5d1a30, user_id=00a810e782154d838e5b73368f3db35e, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:43.793250 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64132fc9-85b9-45ed-9fd0-e6c3d6284ba2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:43.919357 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 903/1820] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:43 2022] PUT /identity/v3/projects/fa5d6717df8147e88a9afaea2a5d1a30/users/00a810e782154d838e5b73368f3db35e/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:43.935650 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b80299-a3e7-4b41-9ba5-9d6f8083cf84 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:43.935650 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b80299-a3e7-4b41-9ba5-9d6f8083cf84 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:43.935650 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b80299-a3e7-4b41-9ba5-9d6f8083cf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:43.939292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-3ed3f4e6-609a-49b5-8704-24946a3cb3ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c46bf6de96964159b25fb03379a6ea88', 'request_id': 'req-3ed3f4e6-609a-49b5-8704-24946a3cb3ef'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:43.948706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-3ed3f4e6-609a-49b5-8704-24946a3cb3ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c46bf6de96964159b25fb03379a6ea88', 'request_id': 'req-3ed3f4e6-609a-49b5-8704-24946a3cb3ef'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:43.962603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-40b80299-a3e7-4b41-9ba5-9d6f8083cf84 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:43.964314 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-40b80299-a3e7-4b41-9ba5-9d6f8083cf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:44.038618 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-3ed3f4e6-609a-49b5-8704-24946a3cb3ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c46bf6de96964159b25fb03379a6ea88', 'request_id': 'req-3ed3f4e6-609a-49b5-8704-24946a3cb3ef'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:44.040953 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 904/1821] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:44.042616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a48e7-c0ba-4676-9c30-eea002ef7bb8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:44.043176 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a48e7-c0ba-4676-9c30-eea002ef7bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:44.043615 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a48e7-c0ba-4676-9c30-eea002ef7bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:44.061132 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-3ed3f4e6-609a-49b5-8704-24946a3cb3ef admin admin] The token cache is being invalidate because project c46bf6de96964159b25fb03379a6ea88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:44.062090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-3ed3f4e6-609a-49b5-8704-24946a3cb3ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:44.062090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-3ed3f4e6-609a-49b5-8704-24946a3cb3ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:44.064606 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 918/1822] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:43 2022] DELETE /identity/v3/projects/c46bf6de96964159b25fb03379a6ea88 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:44.065075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3f4e6-609a-49b5-8704-24946a3cb3ef admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:44.067485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3ed3f4e6-609a-49b5-8704-24946a3cb3ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:44.082361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3f4e6-609a-49b5-8704-24946a3cb3ef admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:44.083377 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-174a48e7-c0ba-4676-9c30-eea002ef7bb8 None None] MFA Rules not processed for user `059d623a01ea48d79fb6fbebdb92551f`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:44.085572 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-174a48e7-c0ba-4676-9c30-eea002ef7bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:44.087930 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd762d5-df4d-4dd5-8ab2-53ae2d63bdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:44.088839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd762d5-df4d-4dd5-8ab2-53ae2d63bdfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:44.090856 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd762d5-df4d-4dd5-8ab2-53ae2d63bdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:44.090856 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd762d5-df4d-4dd5-8ab2-53ae2d63bdfd admin admin] PATH_INFO: `/v3/projects/e4d717cdae144d69ac48fc650e970628` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:44.090856 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd762d5-df4d-4dd5-8ab2-53ae2d63bdfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4d717cdae144d69ac48fc650e970628)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:44.092788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd762d5-df4d-4dd5-8ab2-53ae2d63bdfd admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:44.159876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 905/1823] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:44.161179 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524893e0-aced-4e4d-b83b-86e04743071c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:44.165328 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524893e0-aced-4e4d-b83b-86e04743071c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:44.165328 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524893e0-aced-4e4d-b83b-86e04743071c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:44.183908 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ffd762d5-df4d-4dd5-8ab2-53ae2d63bdfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4d717cdae144d69ac48fc650e970628', 'request_id': 'req-ffd762d5-df4d-4dd5-8ab2-53ae2d63bdfd'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:44.191762 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ffd762d5-df4d-4dd5-8ab2-53ae2d63bdfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4d717cdae144d69ac48fc650e970628', 'request_id': 'req-ffd762d5-df4d-4dd5-8ab2-53ae2d63bdfd'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:44.196954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-524893e0-aced-4e4d-b83b-86e04743071c None None] MFA Rules not processed for user `00a810e782154d838e5b73368f3db35e`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:44.198241 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-524893e0-aced-4e4d-b83b-86e04743071c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:44.260165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ffd762d5-df4d-4dd5-8ab2-53ae2d63bdfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4d717cdae144d69ac48fc650e970628', 'request_id': 'req-ffd762d5-df4d-4dd5-8ab2-53ae2d63bdfd'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:44.266907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 906/1824] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:44.267841 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524893e0-aced-4e4d-b83b-86e04743071c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:44.269365 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-524893e0-aced-4e4d-b83b-86e04743071c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:44.272551 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ffd762d5-df4d-4dd5-8ab2-53ae2d63bdfd admin admin] The token cache is being invalidate because project e4d717cdae144d69ac48fc650e970628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:44.272870 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ffd762d5-df4d-4dd5-8ab2-53ae2d63bdfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:44.273554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ffd762d5-df4d-4dd5-8ab2-53ae2d63bdfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:44.275170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 919/1825] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:44 2022] DELETE /identity/v3/projects/e4d717cdae144d69ac48fc650e970628 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:44.276744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd762d5-df4d-4dd5-8ab2-53ae2d63bdfd admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:44.279482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ffd762d5-df4d-4dd5-8ab2-53ae2d63bdfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:44.340199 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524893e0-aced-4e4d-b83b-86e04743071c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:44.344135 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf4252-2f50-43a7-9add-9fd497c9f075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:44.345226 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf4252-2f50-43a7-9add-9fd497c9f075 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:44.345226 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf4252-2f50-43a7-9add-9fd497c9f075 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:44.345593 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf4252-2f50-43a7-9add-9fd497c9f075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:44.348844 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-becf4252-2f50-43a7-9add-9fd497c9f075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:44.358637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd762d5-df4d-4dd5-8ab2-53ae2d63bdfd admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:44.363151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf4252-2f50-43a7-9add-9fd497c9f075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:44.363644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdc685-b06a-41a6-beba-c0428996fa74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:44.364534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdc685-b06a-41a6-beba-c0428996fa74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:44.364636 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf4252-2f50-43a7-9add-9fd497c9f075 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:44.364959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdc685-b06a-41a6-beba-c0428996fa74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:44.365269 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdc685-b06a-41a6-beba-c0428996fa74 admin admin] PATH_INFO: `/v3/users/2a65035b21a8486ab8878b0d1269be33` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:44.366521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cdc685-b06a-41a6-beba-c0428996fa74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a65035b21a8486ab8878b0d1269be33)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:44.368501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cdc685-b06a-41a6-beba-c0428996fa74 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:44.374833 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 907/1826] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:44.375657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf4252-2f50-43a7-9add-9fd497c9f075 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:44.376950 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-becf4252-2f50-43a7-9add-9fd497c9f075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:44.462134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf4252-2f50-43a7-9add-9fd497c9f075 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:44.471214 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66233cf7-01d0-4416-a445-89824d811328 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:44.471927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66233cf7-01d0-4416-a445-89824d811328 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:44.472217 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66233cf7-01d0-4416-a445-89824d811328 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:44.472506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66233cf7-01d0-4416-a445-89824d811328 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:44.474462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-66233cf7-01d0-4416-a445-89824d811328 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:44.537913 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66233cf7-01d0-4416-a445-89824d811328 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:44.539541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66233cf7-01d0-4416-a445-89824d811328 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:44.548606 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 908/1827] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:44.549418 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66233cf7-01d0-4416-a445-89824d811328 service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:44.550748 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-66233cf7-01d0-4416-a445-89824d811328 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:44.563907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66233cf7-01d0-4416-a445-89824d811328 service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:44.574372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31216f6-235f-42c1-a749-e04e6e2b7eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:44.575338 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31216f6-235f-42c1-a749-e04e6e2b7eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:44.575720 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31216f6-235f-42c1-a749-e04e6e2b7eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:44.576083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31216f6-235f-42c1-a749-e04e6e2b7eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:44.576950 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31216f6-235f-42c1-a749-e04e6e2b7eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:44.579553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31216f6-235f-42c1-a749-e04e6e2b7eb7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:44.626535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-71cdc685-b06a-41a6-beba-c0428996fa74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a65035b21a8486ab8878b0d1269be33'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:44.626535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-71cdc685-b06a-41a6-beba-c0428996fa74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a65035b21a8486ab8878b0d1269be33'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:44.634414 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-71cdc685-b06a-41a6-beba-c0428996fa74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a65035b21a8486ab8878b0d1269be33'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:44.637419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-71cdc685-b06a-41a6-beba-c0428996fa74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a65035b21a8486ab8878b0d1269be33'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:44.669059 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 909/1828] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:44 2022] POST /identity/v3/projects => generated 363 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:44.669751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31216f6-235f-42c1-a749-e04e6e2b7eb7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:44.674561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d31216f6-235f-42c1-a749-e04e6e2b7eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:44.677557 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-71cdc685-b06a-41a6-beba-c0428996fa74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a65035b21a8486ab8878b0d1269be33'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:44.720350 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 920/1829] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:44 2022] DELETE /identity/v3/users/2a65035b21a8486ab8878b0d1269be33 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:44.721557 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdc685-b06a-41a6-beba-c0428996fa74 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:44.727179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-71cdc685-b06a-41a6-beba-c0428996fa74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:44.817820 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:44.839351 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31216f6-235f-42c1-a749-e04e6e2b7eb7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:44.952571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdc685-b06a-41a6-beba-c0428996fa74 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:44.959901 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6a269-e765-4bbd-9bec-f45689663858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:44.961235 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6a269-e765-4bbd-9bec-f45689663858 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:44.961695 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6a269-e765-4bbd-9bec-f45689663858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:44.962120 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6a269-e765-4bbd-9bec-f45689663858 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:44.963018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6a269-e765-4bbd-9bec-f45689663858 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:44.965231 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6a269-e765-4bbd-9bec-f45689663858 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:45.003756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0e7a6-dda3-46c1-9aec-473b58473800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:45.005341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0e7a6-dda3-46c1-9aec-473b58473800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:45.007143 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0e7a6-dda3-46c1-9aec-473b58473800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:45.008613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0e7a6-dda3-46c1-9aec-473b58473800 admin admin] PATH_INFO: `/v3/users/c446abad56be439b8d93e2953f358960` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:45.012160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0e7a6-dda3-46c1-9aec-473b58473800 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c446abad56be439b8d93e2953f358960)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:45.016307 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0e7a6-dda3-46c1-9aec-473b58473800 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:45.066998 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 921/1830] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:44 2022] POST /identity/v3/users => generated 288 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:45.068197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6a269-e765-4bbd-9bec-f45689663858 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:45.070790 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7ef6a269-e765-4bbd-9bec-f45689663858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:45.102609 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6a269-e765-4bbd-9bec-f45689663858 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:45.117943 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf813526-dc3b-4439-968f-f8d0a191965d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:45.117943 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf813526-dc3b-4439-968f-f8d0a191965d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:45.117943 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf813526-dc3b-4439-968f-f8d0a191965d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:45.117943 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf813526-dc3b-4439-968f-f8d0a191965d admin admin] PATH_INFO: `/v3/projects/7d4334cc21564921b3d007b9dfbb9260` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:45.118843 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf813526-dc3b-4439-968f-f8d0a191965d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d4334cc21564921b3d007b9dfbb9260)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:45.122351 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf813526-dc3b-4439-968f-f8d0a191965d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:45.476188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6bb0e7a6-dda3-46c1-9aec-473b58473800 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c446abad56be439b8d93e2953f358960'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:45.477649 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bf813526-dc3b-4439-968f-f8d0a191965d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d4334cc21564921b3d007b9dfbb9260', 'request_id': 'req-bf813526-dc3b-4439-968f-f8d0a191965d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:45.479676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6bb0e7a6-dda3-46c1-9aec-473b58473800 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c446abad56be439b8d93e2953f358960'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:45.493924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:45.495007 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bf813526-dc3b-4439-968f-f8d0a191965d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d4334cc21564921b3d007b9dfbb9260', 'request_id': 'req-bf813526-dc3b-4439-968f-f8d0a191965d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:45.503161 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6bb0e7a6-dda3-46c1-9aec-473b58473800 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c446abad56be439b8d93e2953f358960'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:45.504115 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6bb0e7a6-dda3-46c1-9aec-473b58473800 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c446abad56be439b8d93e2953f358960'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:45.519142 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6bb0e7a6-dda3-46c1-9aec-473b58473800 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c446abad56be439b8d93e2953f358960'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:45.575215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bf813526-dc3b-4439-968f-f8d0a191965d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d4334cc21564921b3d007b9dfbb9260', 'request_id': 'req-bf813526-dc3b-4439-968f-f8d0a191965d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:45.592546 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bf813526-dc3b-4439-968f-f8d0a191965d admin admin] The token cache is being invalidate because project 7d4334cc21564921b3d007b9dfbb9260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:45.592851 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bf813526-dc3b-4439-968f-f8d0a191965d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:45.597809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bf813526-dc3b-4439-968f-f8d0a191965d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:45.602370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 922/1831] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:45 2022] DELETE /identity/v3/projects/7d4334cc21564921b3d007b9dfbb9260 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:45.608300 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf813526-dc3b-4439-968f-f8d0a191965d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:45.612325 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-bf813526-dc3b-4439-968f-f8d0a191965d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:45.642906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 910/1832] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:44 2022] DELETE /identity/v3/users/c446abad56be439b8d93e2953f358960 => generated 0 bytes in 973 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:45.643889 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0e7a6-dda3-46c1-9aec-473b58473800 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:45.649624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6bb0e7a6-dda3-46c1-9aec-473b58473800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:45.697644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf813526-dc3b-4439-968f-f8d0a191965d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:45.742759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0e7a6-dda3-46c1-9aec-473b58473800 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:45.748961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291fa1b-5b66-4222-bb70-1527c6442e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:45.749934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291fa1b-5b66-4222-bb70-1527c6442e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:45.750500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291fa1b-5b66-4222-bb70-1527c6442e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:45.750741 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291fa1b-5b66-4222-bb70-1527c6442e12 admin admin] PATH_INFO: `/v3/users/059d623a01ea48d79fb6fbebdb92551f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:45.752649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291fa1b-5b66-4222-bb70-1527c6442e12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=059d623a01ea48d79fb6fbebdb92551f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:45.755499 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291fa1b-5b66-4222-bb70-1527c6442e12 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:45.781771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788936fb-b135-4c0d-9005-a7fd75d61e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:45.783111 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788936fb-b135-4c0d-9005-a7fd75d61e27 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:45.783616 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788936fb-b135-4c0d-9005-a7fd75d61e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:45.784040 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788936fb-b135-4c0d-9005-a7fd75d61e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:45.785321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788936fb-b135-4c0d-9005-a7fd75d61e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:45.787143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788936fb-b135-4c0d-9005-a7fd75d61e27 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:45.800880 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 923/1833] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:45.802143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f8629-0261-49b1-87b4-ae132da8bd8d None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:45.802607 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f8629-0261-49b1-87b4-ae132da8bd8d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:45.803038 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f8629-0261-49b1-87b4-ae132da8bd8d None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:45.804897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 924/1834] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jun 7 08:38:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:38:45.805910 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f8629-0261-49b1-87b4-ae132da8bd8d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:45.807803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-438f8629-0261-49b1-87b4-ae132da8bd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:45.860960 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f8629-0261-49b1-87b4-ae132da8bd8d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:45.872361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445699d-3043-4b5c-8ae2-726e7a3b091f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:45.873307 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445699d-3043-4b5c-8ae2-726e7a3b091f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:45.873744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445699d-3043-4b5c-8ae2-726e7a3b091f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:45.874041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445699d-3043-4b5c-8ae2-726e7a3b091f admin admin] PATH_INFO: `/v3/users/fc2b283c30fa47b990c14c6fad85412a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:45.875394 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9445699d-3043-4b5c-8ae2-726e7a3b091f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc2b283c30fa47b990c14c6fad85412a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:45.879373 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9445699d-3043-4b5c-8ae2-726e7a3b091f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:45.973652 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2291fa1b-5b66-4222-bb70-1527c6442e12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '059d623a01ea48d79fb6fbebdb92551f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:45.974414 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2291fa1b-5b66-4222-bb70-1527c6442e12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '059d623a01ea48d79fb6fbebdb92551f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:45.987496 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2291fa1b-5b66-4222-bb70-1527c6442e12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '059d623a01ea48d79fb6fbebdb92551f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:46.002886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2291fa1b-5b66-4222-bb70-1527c6442e12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '059d623a01ea48d79fb6fbebdb92551f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:46.011631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2291fa1b-5b66-4222-bb70-1527c6442e12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '059d623a01ea48d79fb6fbebdb92551f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:46.087157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 911/1835] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:45 2022] DELETE /identity/v3/users/059d623a01ea48d79fb6fbebdb92551f => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:46.088350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291fa1b-5b66-4222-bb70-1527c6442e12 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:46.091540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2291fa1b-5b66-4222-bb70-1527c6442e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:46.103438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9445699d-3043-4b5c-8ae2-726e7a3b091f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc2b283c30fa47b990c14c6fad85412a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:46.104907 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9445699d-3043-4b5c-8ae2-726e7a3b091f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc2b283c30fa47b990c14c6fad85412a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:46.116136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9445699d-3043-4b5c-8ae2-726e7a3b091f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc2b283c30fa47b990c14c6fad85412a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:46.116780 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9445699d-3043-4b5c-8ae2-726e7a3b091f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc2b283c30fa47b990c14c6fad85412a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:46.198496 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9445699d-3043-4b5c-8ae2-726e7a3b091f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc2b283c30fa47b990c14c6fad85412a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:46.208622 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291fa1b-5b66-4222-bb70-1527c6442e12 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:46.220655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b76d5-a953-4b86-bda8-d0ae57a9fb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:46.225768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b76d5-a953-4b86-bda8-d0ae57a9fb64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:46.229166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b76d5-a953-4b86-bda8-d0ae57a9fb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:46.229166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b76d5-a953-4b86-bda8-d0ae57a9fb64 admin admin] PATH_INFO: `/v3/projects/c74cd0d8bd3945fb9fbc118ca018f951` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:46.229166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b76d5-a953-4b86-bda8-d0ae57a9fb64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c74cd0d8bd3945fb9fbc118ca018f951)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:46.230056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 925/1836] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:45 2022] DELETE /identity/v3/users/fc2b283c30fa47b990c14c6fad85412a => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:46.231561 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445699d-3043-4b5c-8ae2-726e7a3b091f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:46.232333 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b76d5-a953-4b86-bda8-d0ae57a9fb64 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:46.233471 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9445699d-3043-4b5c-8ae2-726e7a3b091f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:46.249808 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:46.369192 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445699d-3043-4b5c-8ae2-726e7a3b091f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:46.376858 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b2ab9-b15b-486d-8151-7012f8ed4079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:46.379847 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b2ab9-b15b-486d-8151-7012f8ed4079 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:46.380529 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b2ab9-b15b-486d-8151-7012f8ed4079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:46.380991 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b2ab9-b15b-486d-8151-7012f8ed4079 admin admin] PATH_INFO: `/v3/projects/f176de79f5a94ffca08cdac388a31a21/users/276fed1aef864a3f954d4a17ad3ad28b/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:46.407477 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b2ab9-b15b-486d-8151-7012f8ed4079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f176de79f5a94ffca08cdac388a31a21, user_id=276fed1aef864a3f954d4a17ad3ad28b, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:46.413264 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b2ab9-b15b-486d-8151-7012f8ed4079 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:46.565610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 926/1837] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:46 2022] PUT /identity/v3/projects/f176de79f5a94ffca08cdac388a31a21/users/276fed1aef864a3f954d4a17ad3ad28b/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:46.566922 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98ddf6-59a1-4bcf-b564-92631cdd6cbd None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:46.567202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98ddf6-59a1-4bcf-b564-92631cdd6cbd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:46.567628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98ddf6-59a1-4bcf-b564-92631cdd6cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:46.579118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-762b76d5-a953-4b86-bda8-d0ae57a9fb64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c74cd0d8bd3945fb9fbc118ca018f951', 'request_id': 'req-762b76d5-a953-4b86-bda8-d0ae57a9fb64'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:46.586405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-762b76d5-a953-4b86-bda8-d0ae57a9fb64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c74cd0d8bd3945fb9fbc118ca018f951', 'request_id': 'req-762b76d5-a953-4b86-bda8-d0ae57a9fb64'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:46.587879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-4f98ddf6-59a1-4bcf-b564-92631cdd6cbd None None] MFA Rules not processed for user `ee42568b059b49a198ed2c01d5efdafd`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:46.589295 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4f98ddf6-59a1-4bcf-b564-92631cdd6cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:46.594501 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-762b76d5-a953-4b86-bda8-d0ae57a9fb64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c74cd0d8bd3945fb9fbc118ca018f951', 'request_id': 'req-762b76d5-a953-4b86-bda8-d0ae57a9fb64'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:46.709461 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 927/1838] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jun 7 08:38:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:46.713636 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9a59c-4f49-44fd-8d8b-35bf82ad54ca None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:46.713636 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9a59c-4f49-44fd-8d8b-35bf82ad54ca None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:46.713636 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9a59c-4f49-44fd-8d8b-35bf82ad54ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:46.721904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-762b76d5-a953-4b86-bda8-d0ae57a9fb64 admin admin] The token cache is being invalidate because project c74cd0d8bd3945fb9fbc118ca018f951 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:46.722339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-762b76d5-a953-4b86-bda8-d0ae57a9fb64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:46.723753 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-762b76d5-a953-4b86-bda8-d0ae57a9fb64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:46.727320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 912/1839] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:46 2022] DELETE /identity/v3/projects/c74cd0d8bd3945fb9fbc118ca018f951 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:46.730091 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b76d5-a953-4b86-bda8-d0ae57a9fb64 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:46.735873 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-762b76d5-a953-4b86-bda8-d0ae57a9fb64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:46.751501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c1e9a59c-4f49-44fd-8d8b-35bf82ad54ca None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:46.753037 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c1e9a59c-4f49-44fd-8d8b-35bf82ad54ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:46.825648 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 928/1840] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:46.827833 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9a59c-4f49-44fd-8d8b-35bf82ad54ca None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:46.829413 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c1e9a59c-4f49-44fd-8d8b-35bf82ad54ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:46.848679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9a59c-4f49-44fd-8d8b-35bf82ad54ca None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:46.855812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537d358-2275-477a-8aba-7faf33e175ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:46.856595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537d358-2275-477a-8aba-7faf33e175ee service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:46.856757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537d358-2275-477a-8aba-7faf33e175ee service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:46.857030 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537d358-2275-477a-8aba-7faf33e175ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:46.859022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7537d358-2275-477a-8aba-7faf33e175ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:46.872446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537d358-2275-477a-8aba-7faf33e175ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:46.874540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537d358-2275-477a-8aba-7faf33e175ee service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:46.887800 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 929/1841] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:46.889298 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d786afe-223b-4289-b8c1-9ef64b29d157 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:46.889591 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d786afe-223b-4289-b8c1-9ef64b29d157 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:46.889866 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d786afe-223b-4289-b8c1-9ef64b29d157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:46.895814 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b76d5-a953-4b86-bda8-d0ae57a9fb64 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:46.902624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff5d96-3c62-4201-a543-ce81bb12ad4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:46.903317 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff5d96-3c62-4201-a543-ce81bb12ad4a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:46.903599 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff5d96-3c62-4201-a543-ce81bb12ad4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:46.903875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff5d96-3c62-4201-a543-ce81bb12ad4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:46.904608 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff5d96-3c62-4201-a543-ce81bb12ad4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:46.910031 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff5d96-3c62-4201-a543-ce81bb12ad4a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:46.912295 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7d786afe-223b-4289-b8c1-9ef64b29d157 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:46.914624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7d786afe-223b-4289-b8c1-9ef64b29d157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:46.926459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 930/1842] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:46.928898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d786afe-223b-4289-b8c1-9ef64b29d157 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:46.930799 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7d786afe-223b-4289-b8c1-9ef64b29d157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:46.941417 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 913/1843] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:46.943835 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff5d96-3c62-4201-a543-ce81bb12ad4a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:46.946759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:46.948219 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f8ff5d96-3c62-4201-a543-ce81bb12ad4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:47.048125 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff5d96-3c62-4201-a543-ce81bb12ad4a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:47.061237 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a595b9-1d8e-4146-af41-ca3fbbec1f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:47.062245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a595b9-1d8e-4146-af41-ca3fbbec1f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:47.062741 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a595b9-1d8e-4146-af41-ca3fbbec1f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:47.063025 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a595b9-1d8e-4146-af41-ca3fbbec1f3d admin admin] PATH_INFO: `/v3/projects/ca5799381f704770879b32537fe0c8ae` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:47.064316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a595b9-1d8e-4146-af41-ca3fbbec1f3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca5799381f704770879b32537fe0c8ae)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:47.064791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d786afe-223b-4289-b8c1-9ef64b29d157 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:47.067107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a595b9-1d8e-4146-af41-ca3fbbec1f3d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:47.073155 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464b0cf-98d0-4abe-9d6c-cf24f94a47ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:47.076633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464b0cf-98d0-4abe-9d6c-cf24f94a47ca service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:47.076633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464b0cf-98d0-4abe-9d6c-cf24f94a47ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:47.076633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464b0cf-98d0-4abe-9d6c-cf24f94a47ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:47.087691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464b0cf-98d0-4abe-9d6c-cf24f94a47ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:47.089566 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464b0cf-98d0-4abe-9d6c-cf24f94a47ca service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:47.105666 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 931/1844] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:47.107624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78eb73-b1f7-4c4e-b3bf-d2081a0fe1d2 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:47.108443 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78eb73-b1f7-4c4e-b3bf-d2081a0fe1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:47.108884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78eb73-b1f7-4c4e-b3bf-d2081a0fe1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:47.133546 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-8b78eb73-b1f7-4c4e-b3bf-d2081a0fe1d2 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:47.135555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8b78eb73-b1f7-4c4e-b3bf-d2081a0fe1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:47.153658 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 932/1845] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:47.153658 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78eb73-b1f7-4c4e-b3bf-d2081a0fe1d2 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:47.171330 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78eb73-b1f7-4c4e-b3bf-d2081a0fe1d2 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:47.196260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8f8c4-5644-4137-9ebb-8aece46767e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:47.198090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8f8c4-5644-4137-9ebb-8aece46767e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:47.198606 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8f8c4-5644-4137-9ebb-8aece46767e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:47.199389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8f8c4-5644-4137-9ebb-8aece46767e1 admin admin] PATH_INFO: `/v3/projects/f176de79f5a94ffca08cdac388a31a21/users/276fed1aef864a3f954d4a17ad3ad28b/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:47.207970 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8f8c4-5644-4137-9ebb-8aece46767e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f176de79f5a94ffca08cdac388a31a21, user_id=276fed1aef864a3f954d4a17ad3ad28b, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:47.208267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8f8c4-5644-4137-9ebb-8aece46767e1 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:47.234583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 933/1846] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:47 2022] PUT /identity/v3/projects/f176de79f5a94ffca08cdac388a31a21/users/276fed1aef864a3f954d4a17ad3ad28b/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:47.235623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8f8c4-5644-4137-9ebb-8aece46767e1 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:47.238173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-26f8f8c4-5644-4137-9ebb-8aece46767e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:47.293184 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-08a595b9-1d8e-4146-af41-ca3fbbec1f3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca5799381f704770879b32537fe0c8ae', 'request_id': 'req-08a595b9-1d8e-4146-af41-ca3fbbec1f3d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:47.309056 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-08a595b9-1d8e-4146-af41-ca3fbbec1f3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca5799381f704770879b32537fe0c8ae', 'request_id': 'req-08a595b9-1d8e-4146-af41-ca3fbbec1f3d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:47.319262 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-08a595b9-1d8e-4146-af41-ca3fbbec1f3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca5799381f704770879b32537fe0c8ae', 'request_id': 'req-08a595b9-1d8e-4146-af41-ca3fbbec1f3d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:47.349377 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8f8c4-5644-4137-9ebb-8aece46767e1 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:47.419627 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-08a595b9-1d8e-4146-af41-ca3fbbec1f3d admin admin] The token cache is being invalidate because project ca5799381f704770879b32537fe0c8ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:47.420282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-08a595b9-1d8e-4146-af41-ca3fbbec1f3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:47.422531 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-08a595b9-1d8e-4146-af41-ca3fbbec1f3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:47.428071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 914/1847] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:46 2022] DELETE /identity/v3/projects/ca5799381f704770879b32537fe0c8ae => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:47.428071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a595b9-1d8e-4146-af41-ca3fbbec1f3d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:47.430388 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-08a595b9-1d8e-4146-af41-ca3fbbec1f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:47.472351 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cacf4f-4c7f-4f10-bba5-ab23ef11da00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:47.473354 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cacf4f-4c7f-4f10-bba5-ab23ef11da00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:47.473635 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cacf4f-4c7f-4f10-bba5-ab23ef11da00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:47.473961 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cacf4f-4c7f-4f10-bba5-ab23ef11da00 admin admin] PATH_INFO: `/v3/projects/2f84feda4dd243c5b335be949822e693` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:47.476642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cacf4f-4c7f-4f10-bba5-ab23ef11da00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f84feda4dd243c5b335be949822e693)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:47.478902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cacf4f-4c7f-4f10-bba5-ab23ef11da00 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:47.558970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a595b9-1d8e-4146-af41-ca3fbbec1f3d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:47.567370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb03cb1-5a13-469a-bc90-31c5ed7100b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:47.568480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb03cb1-5a13-469a-bc90-31c5ed7100b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:47.569524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb03cb1-5a13-469a-bc90-31c5ed7100b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:47.570126 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb03cb1-5a13-469a-bc90-31c5ed7100b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:47.571069 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb03cb1-5a13-469a-bc90-31c5ed7100b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:47.571601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb03cb1-5a13-469a-bc90-31c5ed7100b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:47.574512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb03cb1-5a13-469a-bc90-31c5ed7100b3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:47.591765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 915/1848] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:47.595130 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb6a76-5d1d-49aa-9a3d-8e0bcc9f89e1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:47.595734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb6a76-5d1d-49aa-9a3d-8e0bcc9f89e1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:47.596570 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb6a76-5d1d-49aa-9a3d-8e0bcc9f89e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:47.651960 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f3cb6a76-5d1d-49aa-9a3d-8e0bcc9f89e1 None None] MFA Rules not processed for user `276fed1aef864a3f954d4a17ad3ad28b`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:47.653795 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f3cb6a76-5d1d-49aa-9a3d-8e0bcc9f89e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:47.679231 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-45cacf4f-4c7f-4f10-bba5-ab23ef11da00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f84feda4dd243c5b335be949822e693', 'request_id': 'req-45cacf4f-4c7f-4f10-bba5-ab23ef11da00'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:47.703330 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-45cacf4f-4c7f-4f10-bba5-ab23ef11da00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f84feda4dd243c5b335be949822e693', 'request_id': 'req-45cacf4f-4c7f-4f10-bba5-ab23ef11da00'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:47.742308 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:47.791431 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-45cacf4f-4c7f-4f10-bba5-ab23ef11da00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f84feda4dd243c5b335be949822e693', 'request_id': 'req-45cacf4f-4c7f-4f10-bba5-ab23ef11da00'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:47.801543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 916/1849] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:47 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:47.804967 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ad1b0-9d16-4736-935f-6a1a261adb14 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:47.806851 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-45cacf4f-4c7f-4f10-bba5-ab23ef11da00 admin admin] The token cache is being invalidate because project 2f84feda4dd243c5b335be949822e693 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:47.807526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ad1b0-9d16-4736-935f-6a1a261adb14 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:47.807850 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-45cacf4f-4c7f-4f10-bba5-ab23ef11da00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:47.808456 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ad1b0-9d16-4736-935f-6a1a261adb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:47.810381 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-45cacf4f-4c7f-4f10-bba5-ab23ef11da00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:47.812598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 934/1850] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:47 2022] DELETE /identity/v3/projects/2f84feda4dd243c5b335be949822e693 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:47.817582 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cacf4f-4c7f-4f10-bba5-ab23ef11da00 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:47.822186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-45cacf4f-4c7f-4f10-bba5-ab23ef11da00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:47.853412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9b8ad1b0-9d16-4736-935f-6a1a261adb14 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:47.855593 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9b8ad1b0-9d16-4736-935f-6a1a261adb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:47.931381 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cacf4f-4c7f-4f10-bba5-ab23ef11da00 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:47.936591 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14645152-0fcc-4f2d-94b2-806b02150e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:47.937346 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14645152-0fcc-4f2d-94b2-806b02150e79 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:47.938240 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14645152-0fcc-4f2d-94b2-806b02150e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:47.938527 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14645152-0fcc-4f2d-94b2-806b02150e79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:47.939352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14645152-0fcc-4f2d-94b2-806b02150e79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:47.942111 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14645152-0fcc-4f2d-94b2-806b02150e79 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:47.980483 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 917/1851] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:47.981405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ad1b0-9d16-4736-935f-6a1a261adb14 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:47.982701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9b8ad1b0-9d16-4736-935f-6a1a261adb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:48.061047 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 935/1852] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:47 2022] POST /identity/v3/projects => generated 385 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:48.062565 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd1be9-88ef-496c-b777-745327f31554 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:48.062565 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd1be9-88ef-496c-b777-745327f31554 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:48.062565 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd1be9-88ef-496c-b777-745327f31554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:48.077659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ad1b0-9d16-4736-935f-6a1a261adb14 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:48.084742 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-22cd1be9-88ef-496c-b777-745327f31554 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:48.086084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-22cd1be9-88ef-496c-b777-745327f31554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:48.145372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efc6c1-cfa9-4d88-aeff-a75f3cd2cf19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:48.146427 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efc6c1-cfa9-4d88-aeff-a75f3cd2cf19 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:48.146713 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efc6c1-cfa9-4d88-aeff-a75f3cd2cf19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:48.147033 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efc6c1-cfa9-4d88-aeff-a75f3cd2cf19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:48.148824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b1efc6c1-cfa9-4d88-aeff-a75f3cd2cf19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:48.153497 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 936/1853] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:48.163321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da37d3-54fc-4c40-902c-e3f66f38e9b1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:48.163639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da37d3-54fc-4c40-902c-e3f66f38e9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:48.163876 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da37d3-54fc-4c40-902c-e3f66f38e9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:48.190015 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c6da37d3-54fc-4c40-902c-e3f66f38e9b1 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:48.191553 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c6da37d3-54fc-4c40-902c-e3f66f38e9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:48.204599 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 937/1854] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:48.205399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da37d3-54fc-4c40-902c-e3f66f38e9b1 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:48.206739 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c6da37d3-54fc-4c40-902c-e3f66f38e9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:48.219091 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da37d3-54fc-4c40-902c-e3f66f38e9b1 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:48.226272 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c93f5-6a78-420f-b11e-0dea6605fe6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:48.226456 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1efc6c1-cfa9-4d88-aeff-a75f3cd2cf19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:48.227199 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c93f5-6a78-420f-b11e-0dea6605fe6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:48.227491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c93f5-6a78-420f-b11e-0dea6605fe6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:48.227770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c93f5-6a78-420f-b11e-0dea6605fe6e admin admin] PATH_INFO: `/v3/projects/1921d0de571f454eb510fbee6aa30eb7` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:48.227876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1efc6c1-cfa9-4d88-aeff-a75f3cd2cf19 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:48.229047 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c93f5-6a78-420f-b11e-0dea6605fe6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1921d0de571f454eb510fbee6aa30eb7)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:48.231010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c93f5-6a78-420f-b11e-0dea6605fe6e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:48.237750 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 918/1855] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:47 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:48.238615 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efc6c1-cfa9-4d88-aeff-a75f3cd2cf19 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:48.249987 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efc6c1-cfa9-4d88-aeff-a75f3cd2cf19 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:48.255555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4f089-b012-4014-afeb-5fdd15a6a59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:48.256394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4f089-b012-4014-afeb-5fdd15a6a59c admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:48.256667 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4f089-b012-4014-afeb-5fdd15a6a59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:48.256938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4f089-b012-4014-afeb-5fdd15a6a59c admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:48.257715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4f089-b012-4014-afeb-5fdd15a6a59c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:48.259636 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4f089-b012-4014-afeb-5fdd15a6a59c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:48.351694 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 919/1856] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:48 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:48.351694 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571ba65-3e4c-40e8-ba37-83393f7c068c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:48.351694 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571ba65-3e4c-40e8-ba37-83393f7c068c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:48.352431 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571ba65-3e4c-40e8-ba37-83393f7c068c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:48.362086 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e01c93f5-6a78-420f-b11e-0dea6605fe6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1921d0de571f454eb510fbee6aa30eb7', 'request_id': 'req-e01c93f5-6a78-420f-b11e-0dea6605fe6e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:48.369203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e01c93f5-6a78-420f-b11e-0dea6605fe6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1921d0de571f454eb510fbee6aa30eb7', 'request_id': 'req-e01c93f5-6a78-420f-b11e-0dea6605fe6e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:48.373522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2571ba65-3e4c-40e8-ba37-83393f7c068c None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:48.374725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2571ba65-3e4c-40e8-ba37-83393f7c068c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:48.442655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 920/1857] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:48.443581 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571ba65-3e4c-40e8-ba37-83393f7c068c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:48.446312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e01c93f5-6a78-420f-b11e-0dea6605fe6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1921d0de571f454eb510fbee6aa30eb7', 'request_id': 'req-e01c93f5-6a78-420f-b11e-0dea6605fe6e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:48.450950 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:48.456462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571ba65-3e4c-40e8-ba37-83393f7c068c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:48.458596 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e01c93f5-6a78-420f-b11e-0dea6605fe6e admin admin] The token cache is being invalidate because project 1921d0de571f454eb510fbee6aa30eb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:48.459341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e01c93f5-6a78-420f-b11e-0dea6605fe6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:48.459846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e01c93f5-6a78-420f-b11e-0dea6605fe6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:48.461333 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae20335-d177-4efa-a545-7c1c085b3316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:48.461663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 938/1858] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:48 2022] DELETE /identity/v3/projects/1921d0de571f454eb510fbee6aa30eb7 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:48.465071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae20335-d177-4efa-a545-7c1c085b3316 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:48.465071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae20335-d177-4efa-a545-7c1c085b3316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:48.465305 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c93f5-6a78-420f-b11e-0dea6605fe6e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:48.465391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae20335-d177-4efa-a545-7c1c085b3316 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:48.465391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae20335-d177-4efa-a545-7c1c085b3316 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:48.465391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae20335-d177-4efa-a545-7c1c085b3316 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:48.465621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae20335-d177-4efa-a545-7c1c085b3316 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:48.467194 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e01c93f5-6a78-420f-b11e-0dea6605fe6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:48.479912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 921/1859] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:48.481495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae20335-d177-4efa-a545-7c1c085b3316 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:48.483721 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1ae20335-d177-4efa-a545-7c1c085b3316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:48.487277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c93f5-6a78-420f-b11e-0dea6605fe6e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:48.495221 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c4a46-4b2c-4bce-bac9-34b3fb30543e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:48.496660 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c4a46-4b2c-4bce-bac9-34b3fb30543e admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:48.497381 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c4a46-4b2c-4bce-bac9-34b3fb30543e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:48.497956 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c4a46-4b2c-4bce-bac9-34b3fb30543e admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:48.499344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c4a46-4b2c-4bce-bac9-34b3fb30543e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:48.503143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c4a46-4b2c-4bce-bac9-34b3fb30543e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:48.518125 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 939/1860] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:48.518975 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c4a46-4b2c-4bce-bac9-34b3fb30543e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:48.520634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-990c4a46-4b2c-4bce-bac9-34b3fb30543e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:48.533972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c4a46-4b2c-4bce-bac9-34b3fb30543e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:48.539611 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887e9f0-50e8-441d-8381-bbaa3f870ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:48.540284 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887e9f0-50e8-441d-8381-bbaa3f870ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:48.540559 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887e9f0-50e8-441d-8381-bbaa3f870ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:48.540851 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887e9f0-50e8-441d-8381-bbaa3f870ab0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:48.541616 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887e9f0-50e8-441d-8381-bbaa3f870ab0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:48.541970 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887e9f0-50e8-441d-8381-bbaa3f870ab0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:48.543594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887e9f0-50e8-441d-8381-bbaa3f870ab0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:48.560479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 940/1861] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:48.560479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785af6c5-ad3b-4c05-9ad6-872a557bd897 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:48.560479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785af6c5-ad3b-4c05-9ad6-872a557bd897 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:48.560479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785af6c5-ad3b-4c05-9ad6-872a557bd897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:48.561149 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae20335-d177-4efa-a545-7c1c085b3316 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:48.564012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23528706-014c-4aa0-b787-813b06eb564a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:48.564860 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23528706-014c-4aa0-b787-813b06eb564a service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:48.565243 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23528706-014c-4aa0-b787-813b06eb564a service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:48.565519 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23528706-014c-4aa0-b787-813b06eb564a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:48.567561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-23528706-014c-4aa0-b787-813b06eb564a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:48.577024 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-785af6c5-ad3b-4c05-9ad6-872a557bd897 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:48.578410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-785af6c5-ad3b-4c05-9ad6-872a557bd897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:48.587273 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 941/1862] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:48.606096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785af6c5-ad3b-4c05-9ad6-872a557bd897 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:48.606096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-785af6c5-ad3b-4c05-9ad6-872a557bd897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:48.606096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785af6c5-ad3b-4c05-9ad6-872a557bd897 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:48.606407 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e925213-3458-4b66-96c5-b64646b5d82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:48.607067 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e925213-3458-4b66-96c5-b64646b5d82d admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:48.607375 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e925213-3458-4b66-96c5-b64646b5d82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:48.607651 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e925213-3458-4b66-96c5-b64646b5d82d admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:48.608345 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e925213-3458-4b66-96c5-b64646b5d82d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:48.610292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e925213-3458-4b66-96c5-b64646b5d82d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:48.651310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23528706-014c-4aa0-b787-813b06eb564a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:48.652741 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23528706-014c-4aa0-b787-813b06eb564a service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:48.661801 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 922/1863] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:48 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:48.663244 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23528706-014c-4aa0-b787-813b06eb564a service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:48.673321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23528706-014c-4aa0-b787-813b06eb564a service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:48.678245 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 942/1864] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:48 2022] POST /identity/v3/projects => generated 381 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:48.678245 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e925213-3458-4b66-96c5-b64646b5d82d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:48.733149 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7ba81-9511-4233-8f7f-5cfe1d779b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:48.733837 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7ba81-9511-4233-8f7f-5cfe1d779b20 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:48.734226 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7ba81-9511-4233-8f7f-5cfe1d779b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:48.734509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7ba81-9511-4233-8f7f-5cfe1d779b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:48.735258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d7ba81-9511-4233-8f7f-5cfe1d779b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:48.736885 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d7ba81-9511-4233-8f7f-5cfe1d779b20 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:48.747894 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 923/1865] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:48.752665 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776752b1-962c-40da-8370-e5ace3e2980b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:48.753128 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776752b1-962c-40da-8370-e5ace3e2980b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:48.753418 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776752b1-962c-40da-8370-e5ace3e2980b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:48.756940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e925213-3458-4b66-96c5-b64646b5d82d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:48.763063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a044b5-2916-42bb-af59-f4560f63ee6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:48.763741 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a044b5-2916-42bb-af59-f4560f63ee6b admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:48.764021 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a044b5-2916-42bb-af59-f4560f63ee6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:48.764297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a044b5-2916-42bb-af59-f4560f63ee6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:48.765011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a044b5-2916-42bb-af59-f4560f63ee6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:48.766963 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a044b5-2916-42bb-af59-f4560f63ee6b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:48.775561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-776752b1-962c-40da-8370-e5ace3e2980b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:48.776824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-776752b1-962c-40da-8370-e5ace3e2980b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:48.786017 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 924/1866] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:48.786862 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776752b1-962c-40da-8370-e5ace3e2980b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:48.801460 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776752b1-962c-40da-8370-e5ace3e2980b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:48.806262 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bf255-bf37-454c-8d99-b82d81929281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:48.806943 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bf255-bf37-454c-8d99-b82d81929281 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:48.807225 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bf255-bf37-454c-8d99-b82d81929281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:48.807505 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bf255-bf37-454c-8d99-b82d81929281 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:48.808199 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bf255-bf37-454c-8d99-b82d81929281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:48.810167 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bf255-bf37-454c-8d99-b82d81929281 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:48.878997 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 943/1867] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:48 2022] POST /identity/v3/projects => generated 393 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:48.880108 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a044b5-2916-42bb-af59-f4560f63ee6b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:48.925710 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 925/1868] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:48 2022] POST /identity/v3/users => generated 291 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:48.926739 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bf255-bf37-454c-8d99-b82d81929281 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:48.943183 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bf255-bf37-454c-8d99-b82d81929281 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:49.015873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a044b5-2916-42bb-af59-f4560f63ee6b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:49.024163 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b56f3-8292-4b42-9202-dfb104944c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:49.025131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b56f3-8292-4b42-9202-dfb104944c31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:49.025567 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b56f3-8292-4b42-9202-dfb104944c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:49.025853 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b56f3-8292-4b42-9202-dfb104944c31 admin admin] PATH_INFO: `/v3/projects/e2baf2dbac8b438aaec4e9ceb078c387/users/7f2f0abc6fac4084a9eaa364497a0b51/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:49.034760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f201658-9fb8-48b3-ba12-740a771ca194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:49.035961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f201658-9fb8-48b3-ba12-740a771ca194 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:49.038306 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f201658-9fb8-48b3-ba12-740a771ca194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:49.038777 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f201658-9fb8-48b3-ba12-740a771ca194 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:49.044523 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f201658-9fb8-48b3-ba12-740a771ca194 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:49.044523 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f201658-9fb8-48b3-ba12-740a771ca194 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:49.044523 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f201658-9fb8-48b3-ba12-740a771ca194 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:49.047933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b56f3-8292-4b42-9202-dfb104944c31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2baf2dbac8b438aaec4e9ceb078c387, user_id=7f2f0abc6fac4084a9eaa364497a0b51, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:49.051860 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b56f3-8292-4b42-9202-dfb104944c31 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:49.067797 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 926/1869] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:49.069526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f201658-9fb8-48b3-ba12-740a771ca194 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:49.082651 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f201658-9fb8-48b3-ba12-740a771ca194 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:49.091495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790f30a-fe4d-4743-98a3-a4bce3e631ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:49.092256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790f30a-fe4d-4743-98a3-a4bce3e631ed admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:49.092611 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790f30a-fe4d-4743-98a3-a4bce3e631ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:49.092973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790f30a-fe4d-4743-98a3-a4bce3e631ed admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:49.093843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790f30a-fe4d-4743-98a3-a4bce3e631ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:49.095850 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790f30a-fe4d-4743-98a3-a4bce3e631ed admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:49.129022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 944/1870] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:48 2022] PUT /identity/v3/projects/e2baf2dbac8b438aaec4e9ceb078c387/users/7f2f0abc6fac4084a9eaa364497a0b51/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:49.139736 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b56f3-8292-4b42-9202-dfb104944c31 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:49.202460 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b56f3-8292-4b42-9202-dfb104944c31 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:49.207936 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bdfd2c-1189-4a80-8770-8d4d59813d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:49.208678 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bdfd2c-1189-4a80-8770-8d4d59813d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:49.209035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bdfd2c-1189-4a80-8770-8d4d59813d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:49.211217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bdfd2c-1189-4a80-8770-8d4d59813d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:49.211217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bdfd2c-1189-4a80-8770-8d4d59813d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:49.212748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bdfd2c-1189-4a80-8770-8d4d59813d1f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:49.224087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 945/1871] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:49.226989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bdfd2c-1189-4a80-8770-8d4d59813d1f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:49.234673 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 927/1872] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:49 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:49.236270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97065045-1b3f-4fb7-a737-12e7dbe926fe None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:49.237028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97065045-1b3f-4fb7-a737-12e7dbe926fe None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:49.238011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97065045-1b3f-4fb7-a737-12e7dbe926fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:49.239216 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bdfd2c-1189-4a80-8770-8d4d59813d1f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:49.244718 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4742e-951d-4943-97dd-0dabe396b5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:49.245296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4742e-951d-4943-97dd-0dabe396b5b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:49.245578 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4742e-951d-4943-97dd-0dabe396b5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:49.245862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4742e-951d-4943-97dd-0dabe396b5b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:49.246682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca4742e-951d-4943-97dd-0dabe396b5b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:49.249022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca4742e-951d-4943-97dd-0dabe396b5b4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:49.270847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-97065045-1b3f-4fb7-a737-12e7dbe926fe None None] MFA Rules not processed for user `7f2f0abc6fac4084a9eaa364497a0b51`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:49.272288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-97065045-1b3f-4fb7-a737-12e7dbe926fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:49.353394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 928/1873] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:49.354408 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97065045-1b3f-4fb7-a737-12e7dbe926fe None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:49.365416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97065045-1b3f-4fb7-a737-12e7dbe926fe None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:49.366462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 946/1874] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:49 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:49.367613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4742e-951d-4943-97dd-0dabe396b5b4 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:49.446995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4742e-951d-4943-97dd-0dabe396b5b4 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:49.447791 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278a925-fc65-432e-9b30-8909cb86774d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:49.448594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278a925-fc65-432e-9b30-8909cb86774d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:49.448884 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278a925-fc65-432e-9b30-8909cb86774d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:49.449221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278a925-fc65-432e-9b30-8909cb86774d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:49.449959 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278a925-fc65-432e-9b30-8909cb86774d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:49.451747 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278a925-fc65-432e-9b30-8909cb86774d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:49.452279 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527cd37-f300-4949-af2a-f17cba918b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:49.453598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527cd37-f300-4949-af2a-f17cba918b81 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:49.454485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527cd37-f300-4949-af2a-f17cba918b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:49.454485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527cd37-f300-4949-af2a-f17cba918b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:49.455049 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4527cd37-f300-4949-af2a-f17cba918b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:49.456954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4527cd37-f300-4949-af2a-f17cba918b81 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:49.467262 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 929/1875] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:49.467262 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278a925-fc65-432e-9b30-8909cb86774d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:49.468912 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 947/1876] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:49.469759 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527cd37-f300-4949-af2a-f17cba918b81 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:49.478383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278a925-fc65-432e-9b30-8909cb86774d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:49.479770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527cd37-f300-4949-af2a-f17cba918b81 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:49.483509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2bed9-82bd-40d3-9a5f-a0c86fb24bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:49.484174 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2bed9-82bd-40d3-9a5f-a0c86fb24bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:49.484482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2bed9-82bd-40d3-9a5f-a0c86fb24bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:49.484770 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2bed9-82bd-40d3-9a5f-a0c86fb24bb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:49.484884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07c7a5-5f6f-4f6b-80a5-cc0234c1ff16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:49.485499 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2bed9-82bd-40d3-9a5f-a0c86fb24bb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:49.485594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07c7a5-5f6f-4f6b-80a5-cc0234c1ff16 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:49.485803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07c7a5-5f6f-4f6b-80a5-cc0234c1ff16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:49.486098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07c7a5-5f6f-4f6b-80a5-cc0234c1ff16 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:49.487091 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07c7a5-5f6f-4f6b-80a5-cc0234c1ff16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:49.487433 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2bed9-82bd-40d3-9a5f-a0c86fb24bb8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:49.488708 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07c7a5-5f6f-4f6b-80a5-cc0234c1ff16 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:49.621480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 948/1877] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:49 2022] POST /identity/v3/users => generated 287 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:49.622500 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07c7a5-5f6f-4f6b-80a5-cc0234c1ff16 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:49.632336 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07c7a5-5f6f-4f6b-80a5-cc0234c1ff16 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:49.637103 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9269ee-a119-4fa5-9018-f4c855842af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:49.637978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9269ee-a119-4fa5-9018-f4c855842af3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:49.638260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9269ee-a119-4fa5-9018-f4c855842af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:49.638536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9269ee-a119-4fa5-9018-f4c855842af3 admin admin] PATH_INFO: `/v3/projects/a2ed08ea659e49f8826b87c2d72f605f/users/2b7afb10450c4c1b8b269e2cc16dab91/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:49.656692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9269ee-a119-4fa5-9018-f4c855842af3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ed08ea659e49f8826b87c2d72f605f, user_id=2b7afb10450c4c1b8b269e2cc16dab91, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:49.660747 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 930/1878] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:49 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:49.660747 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2bed9-82bd-40d3-9a5f-a0c86fb24bb8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:49.660980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9269ee-a119-4fa5-9018-f4c855842af3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:49.726811 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2bed9-82bd-40d3-9a5f-a0c86fb24bb8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:49.731800 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd2d82-fb61-441a-abd8-0722c834eb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:49.732485 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd2d82-fb61-441a-abd8-0722c834eb05 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:49.737387 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd2d82-fb61-441a-abd8-0722c834eb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:49.737387 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd2d82-fb61-441a-abd8-0722c834eb05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:49.737387 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd2d82-fb61-441a-abd8-0722c834eb05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:49.737702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 949/1879] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:49 2022] PUT /identity/v3/projects/a2ed08ea659e49f8826b87c2d72f605f/users/2b7afb10450c4c1b8b269e2cc16dab91/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:49.737801 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd2d82-fb61-441a-abd8-0722c834eb05 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:49.737885 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9269ee-a119-4fa5-9018-f4c855842af3 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:49.751970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 931/1880] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:49.751970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd2d82-fb61-441a-abd8-0722c834eb05 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:49.766737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd2d82-fb61-441a-abd8-0722c834eb05 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:49.821769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9269ee-a119-4fa5-9018-f4c855842af3 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:49.828250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a35aaf-6bd4-43ab-8579-19149654869f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:49.829355 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a35aaf-6bd4-43ab-8579-19149654869f admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:49.830045 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a35aaf-6bd4-43ab-8579-19149654869f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:49.830471 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a35aaf-6bd4-43ab-8579-19149654869f admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:49.831664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a35aaf-6bd4-43ab-8579-19149654869f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:49.834192 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a35aaf-6bd4-43ab-8579-19149654869f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:49.840981 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e677839-d21a-45c6-b887-399f663b7199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:49.841768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e677839-d21a-45c6-b887-399f663b7199 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:49.842118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e677839-d21a-45c6-b887-399f663b7199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:49.842459 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e677839-d21a-45c6-b887-399f663b7199 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:49.843212 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e677839-d21a-45c6-b887-399f663b7199 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:49.845817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e677839-d21a-45c6-b887-399f663b7199 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:49.849900 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 950/1881] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:49.853420 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5599044-a60c-41ce-af89-0f41c7169a45 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:49.853819 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5599044-a60c-41ce-af89-0f41c7169a45 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:49.854103 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5599044-a60c-41ce-af89-0f41c7169a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:49.884749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a5599044-a60c-41ce-af89-0f41c7169a45 None None] MFA Rules not processed for user `2b7afb10450c4c1b8b269e2cc16dab91`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:49.886300 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a5599044-a60c-41ce-af89-0f41c7169a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:49.941759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 932/1882] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:49 2022] POST /identity/v3/users => generated 299 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:49.942712 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e677839-d21a-45c6-b887-399f663b7199 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:49.953719 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e677839-d21a-45c6-b887-399f663b7199 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:49.959592 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d316f6-b126-420e-bb01-b75f85a38b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:49.960485 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d316f6-b126-420e-bb01-b75f85a38b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:49.960818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d316f6-b126-420e-bb01-b75f85a38b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:49.961131 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d316f6-b126-420e-bb01-b75f85a38b15 admin admin] PATH_INFO: `/v3/projects/6977726dec1740d48a80da6bdf81d9b4/users/644a0203dbaa470690cd84862b2d891f/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:49.967492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 951/1883] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:49.968348 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5599044-a60c-41ce-af89-0f41c7169a45 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:49.980492 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d316f6-b126-420e-bb01-b75f85a38b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6977726dec1740d48a80da6bdf81d9b4, user_id=644a0203dbaa470690cd84862b2d891f, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:49.980744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5599044-a60c-41ce-af89-0f41c7169a45 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:49.984033 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d316f6-b126-420e-bb01-b75f85a38b15 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:49.985027 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ab2e2-4bd5-4013-9a23-cd5c5be9c2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:49.985711 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ab2e2-4bd5-4013-9a23-cd5c5be9c2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:49.985994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ab2e2-4bd5-4013-9a23-cd5c5be9c2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:49.986272 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ab2e2-4bd5-4013-9a23-cd5c5be9c2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:49.987002 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ab2e2-4bd5-4013-9a23-cd5c5be9c2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:49.988619 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ab2e2-4bd5-4013-9a23-cd5c5be9c2d4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:50.000063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 952/1884] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:50.001237 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:50.002491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ab2e2-4bd5-4013-9a23-cd5c5be9c2d4 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:50.012309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ab2e2-4bd5-4013-9a23-cd5c5be9c2d4 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:50.017072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81197711-7da5-4e3b-9552-1d18447a9bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:50.017731 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81197711-7da5-4e3b-9552-1d18447a9bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:50.018037 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81197711-7da5-4e3b-9552-1d18447a9bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:50.018298 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81197711-7da5-4e3b-9552-1d18447a9bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:50.019023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81197711-7da5-4e3b-9552-1d18447a9bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:50.020626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81197711-7da5-4e3b-9552-1d18447a9bd2 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:50.031340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 953/1885] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:50.038273 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81197711-7da5-4e3b-9552-1d18447a9bd2 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:50.043029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81197711-7da5-4e3b-9552-1d18447a9bd2 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:50.047813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b68379-610e-4d68-a2be-9ebfa8a6518c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:50.048454 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b68379-610e-4d68-a2be-9ebfa8a6518c admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:50.048736 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b68379-610e-4d68-a2be-9ebfa8a6518c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:50.049037 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b68379-610e-4d68-a2be-9ebfa8a6518c admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:50.049728 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b68379-610e-4d68-a2be-9ebfa8a6518c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:50.051627 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b68379-610e-4d68-a2be-9ebfa8a6518c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:50.060474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 933/1886] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:49 2022] PUT /identity/v3/projects/6977726dec1740d48a80da6bdf81d9b4/users/644a0203dbaa470690cd84862b2d891f/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:50.061620 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d316f6-b126-420e-bb01-b75f85a38b15 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:50.137237 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d316f6-b126-420e-bb01-b75f85a38b15 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:50.145612 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47d478-17fd-43e3-9b8f-210e89f4ea1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:50.147633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47d478-17fd-43e3-9b8f-210e89f4ea1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:50.148116 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47d478-17fd-43e3-9b8f-210e89f4ea1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:50.148577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47d478-17fd-43e3-9b8f-210e89f4ea1b admin admin] PATH_INFO: `/v3/projects/00a507ce611e4d3dbc43a79811cb55a9/users/45c0423c96764cf2a7706dc7a56bc1d5/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:50.182072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47d478-17fd-43e3-9b8f-210e89f4ea1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a507ce611e4d3dbc43a79811cb55a9, user_id=45c0423c96764cf2a7706dc7a56bc1d5, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:50.188003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47d478-17fd-43e3-9b8f-210e89f4ea1b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:50.202770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 954/1887] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:50 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:50.202770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b68379-610e-4d68-a2be-9ebfa8a6518c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:50.266382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 934/1888] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:50 2022] PUT /identity/v3/projects/00a507ce611e4d3dbc43a79811cb55a9/users/45c0423c96764cf2a7706dc7a56bc1d5/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:50.266382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da15de-eb44-409a-ae21-6bf6204bdddb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:50.266382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da15de-eb44-409a-ae21-6bf6204bdddb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:50.266382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da15de-eb44-409a-ae21-6bf6204bdddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:50.301708 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b68379-610e-4d68-a2be-9ebfa8a6518c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:50.324713 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a3da15de-eb44-409a-ae21-6bf6204bdddb None None] MFA Rules not processed for user `644a0203dbaa470690cd84862b2d891f`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:50.325988 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a3da15de-eb44-409a-ae21-6bf6204bdddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:50.375374 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84af16-552b-460a-beeb-e4e7b8e74b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:50.376339 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84af16-552b-460a-beeb-e4e7b8e74b96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:50.376693 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84af16-552b-460a-beeb-e4e7b8e74b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:50.377067 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84af16-552b-460a-beeb-e4e7b8e74b96 admin admin] PATH_INFO: `/v3/projects/195d6601aef54c468ed9d24d291ec82a/users/9b090a32f18a46569a6930d264422a58/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:50.394503 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84af16-552b-460a-beeb-e4e7b8e74b96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=195d6601aef54c468ed9d24d291ec82a, user_id=9b090a32f18a46569a6930d264422a58, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:50.397964 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84af16-552b-460a-beeb-e4e7b8e74b96 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:50.404358 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 935/1889] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:50.405572 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da15de-eb44-409a-ae21-6bf6204bdddb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:50.417759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da15de-eb44-409a-ae21-6bf6204bdddb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:50.422525 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dde902-7663-4b91-bf0f-6b227e114ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:50.423199 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dde902-7663-4b91-bf0f-6b227e114ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:50.423510 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dde902-7663-4b91-bf0f-6b227e114ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:50.423834 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dde902-7663-4b91-bf0f-6b227e114ec8 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:50.424560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dde902-7663-4b91-bf0f-6b227e114ec8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:50.426511 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dde902-7663-4b91-bf0f-6b227e114ec8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:50.500727 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 955/1890] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:50 2022] PUT /identity/v3/projects/195d6601aef54c468ed9d24d291ec82a/users/9b090a32f18a46569a6930d264422a58/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:50.501994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b4e9a-1390-4bd7-b7fc-7b1a706756ed None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:50.502283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b4e9a-1390-4bd7-b7fc-7b1a706756ed None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:50.502556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b4e9a-1390-4bd7-b7fc-7b1a706756ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:50.537084 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 936/1891] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:50 2022] POST /identity/v3/users => generated 296 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:50.537084 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dde902-7663-4b91-bf0f-6b227e114ec8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:50.539443 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-fa9b4e9a-1390-4bd7-b7fc-7b1a706756ed None None] MFA Rules not processed for user `45c0423c96764cf2a7706dc7a56bc1d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:50.540896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-fa9b4e9a-1390-4bd7-b7fc-7b1a706756ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:50.547674 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dde902-7663-4b91-bf0f-6b227e114ec8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:50.592053 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:50.643266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd3d45-a334-48b0-b74f-cba40353cefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:50.643266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd3d45-a334-48b0-b74f-cba40353cefc admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:50.643266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd3d45-a334-48b0-b74f-cba40353cefc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:50.643266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd3d45-a334-48b0-b74f-cba40353cefc admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:50.645634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd3d45-a334-48b0-b74f-cba40353cefc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:50.645634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd3d45-a334-48b0-b74f-cba40353cefc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:50.658615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 956/1892] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:50.659939 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b4e9a-1390-4bd7-b7fc-7b1a706756ed None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:50.671592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b4e9a-1390-4bd7-b7fc-7b1a706756ed None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:50.676620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b8236-4a5d-440e-b655-cef121e7aeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:50.677432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b8236-4a5d-440e-b655-cef121e7aeb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:50.677779 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b8236-4a5d-440e-b655-cef121e7aeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:50.678103 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b8236-4a5d-440e-b655-cef121e7aeb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:50.678921 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b8236-4a5d-440e-b655-cef121e7aeb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:50.680747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b8236-4a5d-440e-b655-cef121e7aeb8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:50.703844 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 957/1893] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:50.704810 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b8236-4a5d-440e-b655-cef121e7aeb8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:50.718082 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b8236-4a5d-440e-b655-cef121e7aeb8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:50.723056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98525ef0-4564-4ec1-bcdd-0abe429695b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:50.723896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98525ef0-4564-4ec1-bcdd-0abe429695b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:50.724190 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98525ef0-4564-4ec1-bcdd-0abe429695b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:50.724472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98525ef0-4564-4ec1-bcdd-0abe429695b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:50.725265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98525ef0-4564-4ec1-bcdd-0abe429695b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:50.726916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98525ef0-4564-4ec1-bcdd-0abe429695b1 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:50.742500 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 958/1894] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:50.742500 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98525ef0-4564-4ec1-bcdd-0abe429695b1 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:50.752786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98525ef0-4564-4ec1-bcdd-0abe429695b1 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:50.759799 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f9f80-f330-4264-acca-eeebd95bf43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:50.760499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f9f80-f330-4264-acca-eeebd95bf43e admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:50.760846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f9f80-f330-4264-acca-eeebd95bf43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:50.761099 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f9f80-f330-4264-acca-eeebd95bf43e admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:50.780474 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f9f80-f330-4264-acca-eeebd95bf43e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:50.780474 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f9f80-f330-4264-acca-eeebd95bf43e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:50.832670 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 937/1895] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:50 2022] POST /identity/v3/projects => generated 393 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:50.834217 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd3d45-a334-48b0-b74f-cba40353cefc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:50.896906 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 959/1896] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:50 2022] POST /identity/v3/projects => generated 371 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:50.897948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f9f80-f330-4264-acca-eeebd95bf43e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:50.928606 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd3d45-a334-48b0-b74f-cba40353cefc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:50.966703 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f9f80-f330-4264-acca-eeebd95bf43e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:50.972104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843418d9-3214-4e2f-8ecd-e8d2dbaa44c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:50.973042 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843418d9-3214-4e2f-8ecd-e8d2dbaa44c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:50.973587 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843418d9-3214-4e2f-8ecd-e8d2dbaa44c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:50.973587 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843418d9-3214-4e2f-8ecd-e8d2dbaa44c8 admin admin] PATH_INFO: `/v3/projects/2c204eebfd474bd1b5bff4e1da4cae26/users/e34d059b3a214b6794bac92c4d07d991/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:50.997510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843418d9-3214-4e2f-8ecd-e8d2dbaa44c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c204eebfd474bd1b5bff4e1da4cae26, user_id=e34d059b3a214b6794bac92c4d07d991, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:51.001044 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843418d9-3214-4e2f-8ecd-e8d2dbaa44c8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:51.011519 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69e9cb-496b-43e3-a20b-6319f8bd0f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:51.012424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69e9cb-496b-43e3-a20b-6319f8bd0f1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:51.012704 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69e9cb-496b-43e3-a20b-6319f8bd0f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:51.013071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69e9cb-496b-43e3-a20b-6319f8bd0f1f admin admin] PATH_INFO: `/v3/projects/195d6601aef54c468ed9d24d291ec82a/users/9b090a32f18a46569a6930d264422a58/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:51.015198 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69e9cb-496b-43e3-a20b-6319f8bd0f1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=195d6601aef54c468ed9d24d291ec82a, user_id=9b090a32f18a46569a6930d264422a58, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:51.018035 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69e9cb-496b-43e3-a20b-6319f8bd0f1f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:51.029103 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 938/1897] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:50 2022] PUT /identity/v3/projects/195d6601aef54c468ed9d24d291ec82a/users/9b090a32f18a46569a6930d264422a58/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:51.029918 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69e9cb-496b-43e3-a20b-6319f8bd0f1f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:51.092651 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69e9cb-496b-43e3-a20b-6319f8bd0f1f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:51.097469 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4623f72-5b02-423c-848f-37668a5be6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:51.098138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4623f72-5b02-423c-848f-37668a5be6f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:51.098420 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4623f72-5b02-423c-848f-37668a5be6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:51.098789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4623f72-5b02-423c-848f-37668a5be6f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:51.099618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4623f72-5b02-423c-848f-37668a5be6f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:51.102396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4623f72-5b02-423c-848f-37668a5be6f6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:51.110084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 960/1898] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:50 2022] PUT /identity/v3/projects/2c204eebfd474bd1b5bff4e1da4cae26/users/e34d059b3a214b6794bac92c4d07d991/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:51.110084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843418d9-3214-4e2f-8ecd-e8d2dbaa44c8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:51.191205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843418d9-3214-4e2f-8ecd-e8d2dbaa44c8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:51.192321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 939/1899] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:51 2022] POST /identity/v3/users => generated 303 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:51.195927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f3195-bb5a-45e0-92ca-ec5f5b8c555f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:51.196321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f3195-bb5a-45e0-92ca-ec5f5b8c555f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:51.203027 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f3195-bb5a-45e0-92ca-ec5f5b8c555f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:51.203143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cbf868-cb79-4d64-9554-f18c8f302997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:51.203143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbf868-cb79-4d64-9554-f18c8f302997 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:51.203143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbf868-cb79-4d64-9554-f18c8f302997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:51.203143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbf868-cb79-4d64-9554-f18c8f302997 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:51.203485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cbf868-cb79-4d64-9554-f18c8f302997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:51.203485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cbf868-cb79-4d64-9554-f18c8f302997 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:51.228368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d98f3195-bb5a-45e0-92ca-ec5f5b8c555f None None] MFA Rules not processed for user `9b090a32f18a46569a6930d264422a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:51.229620 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d98f3195-bb5a-45e0-92ca-ec5f5b8c555f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:51.295803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 961/1900] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:51 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:51.296585 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cbf868-cb79-4d64-9554-f18c8f302997 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:51.306824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cbf868-cb79-4d64-9554-f18c8f302997 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:51.307765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 940/1901] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:51.310094 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f3195-bb5a-45e0-92ca-ec5f5b8c555f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:51.322853 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f3195-bb5a-45e0-92ca-ec5f5b8c555f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:51.323028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115b5a2-b32d-479c-ace7-207dddfee251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:51.323577 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115b5a2-b32d-479c-ace7-207dddfee251 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:51.323857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115b5a2-b32d-479c-ace7-207dddfee251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:51.324131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115b5a2-b32d-479c-ace7-207dddfee251 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:51.325153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115b5a2-b32d-479c-ace7-207dddfee251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:51.326813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115b5a2-b32d-479c-ace7-207dddfee251 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:51.327848 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3942eb6-afa7-44c4-9777-90ac77323543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:51.328545 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3942eb6-afa7-44c4-9777-90ac77323543 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:51.329493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3942eb6-afa7-44c4-9777-90ac77323543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:51.329493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3942eb6-afa7-44c4-9777-90ac77323543 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:51.329859 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3942eb6-afa7-44c4-9777-90ac77323543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:51.332142 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3942eb6-afa7-44c4-9777-90ac77323543 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:51.341677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 962/1902] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:51.342625 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115b5a2-b32d-479c-ace7-207dddfee251 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:51.356586 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115b5a2-b32d-479c-ace7-207dddfee251 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:51.362851 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3b8f3-4e69-4339-a0dc-3f1fe444c859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:51.366154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 941/1903] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:51.366499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3b8f3-4e69-4339-a0dc-3f1fe444c859 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:51.366499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3b8f3-4e69-4339-a0dc-3f1fe444c859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:51.366499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3b8f3-4e69-4339-a0dc-3f1fe444c859 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:51.366499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3b8f3-4e69-4339-a0dc-3f1fe444c859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:51.368121 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3b8f3-4e69-4339-a0dc-3f1fe444c859 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:51.368388 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3942eb6-afa7-44c4-9777-90ac77323543 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:51.380770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 963/1904] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:51.382556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3b8f3-4e69-4339-a0dc-3f1fe444c859 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:51.397929 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3b8f3-4e69-4339-a0dc-3f1fe444c859 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:51.408360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb304cf1-d8df-4364-9c92-60d14a3c6dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:51.409566 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb304cf1-d8df-4364-9c92-60d14a3c6dd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:51.409892 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb304cf1-d8df-4364-9c92-60d14a3c6dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:51.410170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb304cf1-d8df-4364-9c92-60d14a3c6dd1 admin admin] PATH_INFO: `/v3/projects/2c204eebfd474bd1b5bff4e1da4cae26/users/e34d059b3a214b6794bac92c4d07d991/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:51.412288 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb304cf1-d8df-4364-9c92-60d14a3c6dd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c204eebfd474bd1b5bff4e1da4cae26, user_id=e34d059b3a214b6794bac92c4d07d991, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:51.415334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb304cf1-d8df-4364-9c92-60d14a3c6dd1 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:51.442817 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 964/1905] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:51 2022] PUT /identity/v3/projects/2c204eebfd474bd1b5bff4e1da4cae26/users/e34d059b3a214b6794bac92c4d07d991/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:51.444921 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb304cf1-d8df-4364-9c92-60d14a3c6dd1 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:51.523461 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3942eb6-afa7-44c4-9777-90ac77323543 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:51.548148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb304cf1-d8df-4364-9c92-60d14a3c6dd1 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:51.556228 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea0ac0-7667-4e5f-ab9a-6ad9e215e9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:51.557778 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea0ac0-7667-4e5f-ab9a-6ad9e215e9d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:51.558169 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea0ac0-7667-4e5f-ab9a-6ad9e215e9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:51.558689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea0ac0-7667-4e5f-ab9a-6ad9e215e9d2 admin admin] PATH_INFO: `/v3/projects/96f5891b5086440ca658e684bcc25063/users/339cdb82fac5405695ce89f6354d00e6/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:51.587929 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea0ac0-7667-4e5f-ab9a-6ad9e215e9d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96f5891b5086440ca658e684bcc25063, user_id=339cdb82fac5405695ce89f6354d00e6, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:51.591124 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea0ac0-7667-4e5f-ab9a-6ad9e215e9d2 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:51.602982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148c67d-2448-41b8-8857-e1153aaf74bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:51.603773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148c67d-2448-41b8-8857-e1153aaf74bb service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:51.604132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148c67d-2448-41b8-8857-e1153aaf74bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:51.604840 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148c67d-2448-41b8-8857-e1153aaf74bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:51.661597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 965/1906] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:51 2022] PUT /identity/v3/projects/96f5891b5086440ca658e684bcc25063/users/339cdb82fac5405695ce89f6354d00e6/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:51.662445 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea0ac0-7667-4e5f-ab9a-6ad9e215e9d2 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:51.689390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148c67d-2448-41b8-8857-e1153aaf74bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:51.690899 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148c67d-2448-41b8-8857-e1153aaf74bb service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:51.701951 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 942/1907] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:51.703304 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba1b5f-0c38-43de-a527-43f9c712a388 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:51.703587 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba1b5f-0c38-43de-a527-43f9c712a388 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:51.703870 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba1b5f-0c38-43de-a527-43f9c712a388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:51.736713 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b7ba1b5f-0c38-43de-a527-43f9c712a388 None None] MFA Rules not processed for user `e34d059b3a214b6794bac92c4d07d991`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:51.738151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b7ba1b5f-0c38-43de-a527-43f9c712a388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:51.755135 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea0ac0-7667-4e5f-ab9a-6ad9e215e9d2 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:51.761108 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08001ec2-226a-45c4-99a2-e650743fa678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:51.762493 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08001ec2-226a-45c4-99a2-e650743fa678 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:51.763069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08001ec2-226a-45c4-99a2-e650743fa678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:51.763392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08001ec2-226a-45c4-99a2-e650743fa678 admin admin] PATH_INFO: `/v3/projects/13ecc7f298a84c5f9282d6ca5bd9b972/users/b0dfc1322cbe4f8086174f0694016710/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:51.796378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08001ec2-226a-45c4-99a2-e650743fa678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13ecc7f298a84c5f9282d6ca5bd9b972, user_id=b0dfc1322cbe4f8086174f0694016710, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:51.800242 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08001ec2-226a-45c4-99a2-e650743fa678 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:51.843557 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 943/1908] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:51.844506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba1b5f-0c38-43de-a527-43f9c712a388 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:51.857684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba1b5f-0c38-43de-a527-43f9c712a388 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:51.863664 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35cbaf-5ab6-4060-a03b-c69068d2db7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:51.864363 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35cbaf-5ab6-4060-a03b-c69068d2db7f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:51.864655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35cbaf-5ab6-4060-a03b-c69068d2db7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:51.864985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35cbaf-5ab6-4060-a03b-c69068d2db7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:51.866383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35cbaf-5ab6-4060-a03b-c69068d2db7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:51.869030 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35cbaf-5ab6-4060-a03b-c69068d2db7f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:51.880009 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 944/1909] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:51.883746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35cbaf-5ab6-4060-a03b-c69068d2db7f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:51.897201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 966/1910] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:51 2022] PUT /identity/v3/projects/13ecc7f298a84c5f9282d6ca5bd9b972/users/b0dfc1322cbe4f8086174f0694016710/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:51.898488 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08001ec2-226a-45c4-99a2-e650743fa678 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:51.921888 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:51.958515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35cbaf-5ab6-4060-a03b-c69068d2db7f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:51.985233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08001ec2-226a-45c4-99a2-e650743fa678 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:51.993367 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca2fc7-c5d8-40bf-a0fc-cc409aff0254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:51.995499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca2fc7-c5d8-40bf-a0fc-cc409aff0254 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:51.996002 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca2fc7-c5d8-40bf-a0fc-cc409aff0254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:51.996909 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca2fc7-c5d8-40bf-a0fc-cc409aff0254 admin admin] PATH_INFO: `/v3/projects/96f5891b5086440ca658e684bcc25063/users/339cdb82fac5405695ce89f6354d00e6/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:51.999475 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca2fc7-c5d8-40bf-a0fc-cc409aff0254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96f5891b5086440ca658e684bcc25063, user_id=339cdb82fac5405695ce89f6354d00e6, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:52.003682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca2fc7-c5d8-40bf-a0fc-cc409aff0254 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:52.016724 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 967/1911] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:51 2022] PUT /identity/v3/projects/96f5891b5086440ca658e684bcc25063/users/339cdb82fac5405695ce89f6354d00e6/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:52.019123 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797bdf31-aa1f-4f66-ac0a-dc2c853de860 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:52.019863 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797bdf31-aa1f-4f66-ac0a-dc2c853de860 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:52.020171 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797bdf31-aa1f-4f66-ac0a-dc2c853de860 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:52.020446 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797bdf31-aa1f-4f66-ac0a-dc2c853de860 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:52.027909 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca2fc7-c5d8-40bf-a0fc-cc409aff0254 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:52.089573 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797bdf31-aa1f-4f66-ac0a-dc2c853de860 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:52.090936 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797bdf31-aa1f-4f66-ac0a-dc2c853de860 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:52.099532 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca2fc7-c5d8-40bf-a0fc-cc409aff0254 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:52.101378 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 945/1912] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:52.104362 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbca327-5b3d-4be1-b489-313cd6585009 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:52.104362 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbca327-5b3d-4be1-b489-313cd6585009 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:52.104362 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbca327-5b3d-4be1-b489-313cd6585009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:52.106983 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698931ed-8290-4cad-8bb4-73cfad94cbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:52.107670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698931ed-8290-4cad-8bb4-73cfad94cbaf admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:52.107959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698931ed-8290-4cad-8bb4-73cfad94cbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:52.108322 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698931ed-8290-4cad-8bb4-73cfad94cbaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:52.109136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698931ed-8290-4cad-8bb4-73cfad94cbaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:52.110742 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698931ed-8290-4cad-8bb4-73cfad94cbaf admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:52.126320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 968/1913] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:52.130212 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698931ed-8290-4cad-8bb4-73cfad94cbaf admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:52.141813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698931ed-8290-4cad-8bb4-73cfad94cbaf admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:52.142959 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ecbca327-5b3d-4be1-b489-313cd6585009 None None] MFA Rules not processed for user `339cdb82fac5405695ce89f6354d00e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:52.144378 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ecbca327-5b3d-4be1-b489-313cd6585009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:52.147399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd170f6a-74f5-42a3-bb5f-3e05f1ff6ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:52.163151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd170f6a-74f5-42a3-bb5f-3e05f1ff6ddf admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:52.163151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd170f6a-74f5-42a3-bb5f-3e05f1ff6ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:52.163151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd170f6a-74f5-42a3-bb5f-3e05f1ff6ddf admin admin] PATH_INFO: `/v3/projects/13ecc7f298a84c5f9282d6ca5bd9b972/users/b0dfc1322cbe4f8086174f0694016710/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:52.163151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd170f6a-74f5-42a3-bb5f-3e05f1ff6ddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=13ecc7f298a84c5f9282d6ca5bd9b972, user_id=b0dfc1322cbe4f8086174f0694016710, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:52.163151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd170f6a-74f5-42a3-bb5f-3e05f1ff6ddf admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:52.195669 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 969/1914] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:52 2022] PUT /identity/v3/projects/13ecc7f298a84c5f9282d6ca5bd9b972/users/b0dfc1322cbe4f8086174f0694016710/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:52.206067 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b0c26-651c-47e7-adbf-66349a07f8b8 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:52.206525 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b0c26-651c-47e7-adbf-66349a07f8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:52.206802 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b0c26-651c-47e7-adbf-66349a07f8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:52.235831 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 946/1915] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:52.242528 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-899b0c26-651c-47e7-adbf-66349a07f8b8 None None] MFA Rules not processed for user `b0dfc1322cbe4f8086174f0694016710`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:52.243853 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-899b0c26-651c-47e7-adbf-66349a07f8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:52.319853 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 970/1916] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:52 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:52.426512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:52.782406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbca327-5b3d-4be1-b489-313cd6585009 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:52.783832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ecbca327-5b3d-4be1-b489-313cd6585009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:52.853553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbca327-5b3d-4be1-b489-313cd6585009 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:52.887051 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:52.887330 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b0c26-651c-47e7-adbf-66349a07f8b8 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:52.920983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b14c2-4977-4499-9bd3-f06ee9f6257a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:52.921709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b14c2-4977-4499-9bd3-f06ee9f6257a service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:52.922001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b14c2-4977-4499-9bd3-f06ee9f6257a service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:52.922294 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b14c2-4977-4499-9bd3-f06ee9f6257a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:52.950585 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b0c26-651c-47e7-adbf-66349a07f8b8 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:52.954665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50377f3-658e-4b43-9ba3-5cf47470f383 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:52.955101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50377f3-658e-4b43-9ba3-5cf47470f383 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:52.955387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50377f3-658e-4b43-9ba3-5cf47470f383 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:52.955677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50377f3-658e-4b43-9ba3-5cf47470f383 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:52.968412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50377f3-658e-4b43-9ba3-5cf47470f383 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:52.970293 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50377f3-658e-4b43-9ba3-5cf47470f383 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:52.985757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 971/1917] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:52 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:38:52.988360 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b14c2-4977-4499-9bd3-f06ee9f6257a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:52.989861 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b14c2-4977-4499-9bd3-f06ee9f6257a service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:53.002163 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 947/1918] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:53.026719 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:53.102119 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:53.313658 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50377f3-658e-4b43-9ba3-5cf47470f383 service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:53.319607 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50377f3-658e-4b43-9ba3-5cf47470f383 service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:53.323557 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d24763-42af-4a67-8b0c-a7ea7f93fccd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:53.324205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d24763-42af-4a67-8b0c-a7ea7f93fccd service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:53.324487 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d24763-42af-4a67-8b0c-a7ea7f93fccd service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:53.324762 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d24763-42af-4a67-8b0c-a7ea7f93fccd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:53.388020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d24763-42af-4a67-8b0c-a7ea7f93fccd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:53.389564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d24763-42af-4a67-8b0c-a7ea7f93fccd service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:53.399120 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 972/1919] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:54.188308 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b14c2-4977-4499-9bd3-f06ee9f6257a service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:54.198441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b14c2-4977-4499-9bd3-f06ee9f6257a service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:54.201453 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2fb61-bd60-4bd0-9933-37bd38593f4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:54.202146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2fb61-bd60-4bd0-9933-37bd38593f4f service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:54.202430 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2fb61-bd60-4bd0-9933-37bd38593f4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:54.202713 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2fb61-bd60-4bd0-9933-37bd38593f4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:54.267282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2fb61-bd60-4bd0-9933-37bd38593f4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:54.268897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2fb61-bd60-4bd0-9933-37bd38593f4f service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:54.280023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 948/1920] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:54.323655 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:54.535483 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:54.565492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d24763-42af-4a67-8b0c-a7ea7f93fccd service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:54.566951 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e0d24763-42af-4a67-8b0c-a7ea7f93fccd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:54.673422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d24763-42af-4a67-8b0c-a7ea7f93fccd service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:54.685814 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf0e9e-9098-4efb-a7a9-3c0f1753130d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:54.685814 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf0e9e-9098-4efb-a7a9-3c0f1753130d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:54.685814 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf0e9e-9098-4efb-a7a9-3c0f1753130d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:54.685814 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf0e9e-9098-4efb-a7a9-3c0f1753130d admin admin] PATH_INFO: `/v3/users/00a810e782154d838e5b73368f3db35e` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:54.686086 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf0e9e-9098-4efb-a7a9-3c0f1753130d admin admin] RBAC: Authorizing `identity:delete_user(user_id=00a810e782154d838e5b73368f3db35e)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:54.691165 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2fb61-bd60-4bd0-9933-37bd38593f4f service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:54.692528 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf0e9e-9098-4efb-a7a9-3c0f1753130d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:54.795780 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2fb61-bd60-4bd0-9933-37bd38593f4f service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:54.801076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7d2b1-d2ab-4ebd-96dd-b111999316b6 tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34d059b3a214b6794bac92c4d07d991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c204eebfd474bd1b5bff4e1da4cae26', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:54.801936 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7d2b1-d2ab-4ebd-96dd-b111999316b6 tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:54.802354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7d2b1-d2ab-4ebd-96dd-b111999316b6 tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:54.802748 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7d2b1-d2ab-4ebd-96dd-b111999316b6 tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:54.804184 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7d2b1-d2ab-4ebd-96dd-b111999316b6 tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:54.804577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7d2b1-d2ab-4ebd-96dd-b111999316b6 tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:54.806420 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7d2b1-d2ab-4ebd-96dd-b111999316b6 tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:54.820422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 949/1921] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:54.830006 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7d2b1-d2ab-4ebd-96dd-b111999316b6 tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:54.835786 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7d2b1-d2ab-4ebd-96dd-b111999316b6 tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:54.841048 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aea860-a633-460d-89e3-ef4d7c3baeda tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34d059b3a214b6794bac92c4d07d991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c204eebfd474bd1b5bff4e1da4cae26', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:54.841971 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aea860-a633-460d-89e3-ef4d7c3baeda tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:54.842328 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aea860-a633-460d-89e3-ef4d7c3baeda tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:54.842686 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aea860-a633-460d-89e3-ef4d7c3baeda tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:54.843463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aea860-a633-460d-89e3-ef4d7c3baeda tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:54.845572 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aea860-a633-460d-89e3-ef4d7c3baeda tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:54.972364 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 950/1922] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:54 2022] POST /identity/v3/projects => generated 353 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:38:54.976863 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aea860-a633-460d-89e3-ef4d7c3baeda tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:55.048340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-13bf0e9e-9098-4efb-a7a9-3c0f1753130d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00a810e782154d838e5b73368f3db35e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:55.058502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-13bf0e9e-9098-4efb-a7a9-3c0f1753130d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00a810e782154d838e5b73368f3db35e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:55.058700 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aea860-a633-460d-89e3-ef4d7c3baeda tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:55.059170 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e2b81-cde9-4423-9d98-a86dbc9c872b tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34d059b3a214b6794bac92c4d07d991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c204eebfd474bd1b5bff4e1da4cae26', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:55.059651 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-13bf0e9e-9098-4efb-a7a9-3c0f1753130d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00a810e782154d838e5b73368f3db35e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:55.060222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e2b81-cde9-4423-9d98-a86dbc9c872b tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:55.060329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-13bf0e9e-9098-4efb-a7a9-3c0f1753130d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00a810e782154d838e5b73368f3db35e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:55.060708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e2b81-cde9-4423-9d98-a86dbc9c872b tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:55.061115 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e2b81-cde9-4423-9d98-a86dbc9c872b tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:55.062496 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e2b81-cde9-4423-9d98-a86dbc9c872b tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:55.063404 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e2b81-cde9-4423-9d98-a86dbc9c872b tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:55.065813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e2b81-cde9-4423-9d98-a86dbc9c872b tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:55.078330 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 951/1923] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:38:55.125583 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e2b81-cde9-4423-9d98-a86dbc9c872b tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:55.127828 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-654e2b81-cde9-4423-9d98-a86dbc9c872b tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:55.142350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e2b81-cde9-4423-9d98-a86dbc9c872b tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:55.156056 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6dfac9-273b-4cf6-9c25-d949d1c6c65a tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34d059b3a214b6794bac92c4d07d991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c204eebfd474bd1b5bff4e1da4cae26', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:55.156056 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6dfac9-273b-4cf6-9c25-d949d1c6c65a tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:55.156056 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6dfac9-273b-4cf6-9c25-d949d1c6c65a tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:55.156504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6dfac9-273b-4cf6-9c25-d949d1c6c65a tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] PATH_INFO: `/v3/projects/bc0737c767834848910b2d8b4c10f270` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:55.156504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6dfac9-273b-4cf6-9c25-d949d1c6c65a tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bc0737c767834848910b2d8b4c10f270)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:55.156504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6dfac9-273b-4cf6-9c25-d949d1c6c65a tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:55.208837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-13bf0e9e-9098-4efb-a7a9-3c0f1753130d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00a810e782154d838e5b73368f3db35e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:55.229647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 973/1924] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:54 2022] DELETE /identity/v3/users/00a810e782154d838e5b73368f3db35e => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:55.234262 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cef56-0d48-4631-9a0f-ab1799a93568 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:55.234550 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cef56-0d48-4631-9a0f-ab1799a93568 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:55.234830 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cef56-0d48-4631-9a0f-ab1799a93568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:55.254208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1c4cef56-0d48-4631-9a0f-ab1799a93568 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:55.255556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1c4cef56-0d48-4631-9a0f-ab1799a93568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:55.324927 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 974/1925] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:38:55.331793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4cef56-0d48-4631-9a0f-ab1799a93568 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:55.333324 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1c4cef56-0d48-4631-9a0f-ab1799a93568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:55.346178 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ca6dfac9-273b-4cf6-9c25-d949d1c6c65a tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc0737c767834848910b2d8b4c10f270', 'request_id': 'req-ca6dfac9-273b-4cf6-9c25-d949d1c6c65a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:55.357702 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ca6dfac9-273b-4cf6-9c25-d949d1c6c65a tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc0737c767834848910b2d8b4c10f270', 'request_id': 'req-ca6dfac9-273b-4cf6-9c25-d949d1c6c65a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:55.365131 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ca6dfac9-273b-4cf6-9c25-d949d1c6c65a tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc0737c767834848910b2d8b4c10f270', 'request_id': 'req-ca6dfac9-273b-4cf6-9c25-d949d1c6c65a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:55.405409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4cef56-0d48-4631-9a0f-ab1799a93568 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:55.435069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:55.444000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ca6dfac9-273b-4cf6-9c25-d949d1c6c65a tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] The token cache is being invalidate because project bc0737c767834848910b2d8b4c10f270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:55.444294 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ca6dfac9-273b-4cf6-9c25-d949d1c6c65a tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:55.445197 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ca6dfac9-273b-4cf6-9c25-d949d1c6c65a tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:55.447144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 952/1926] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:55 2022] DELETE /identity/v3/projects/bc0737c767834848910b2d8b4c10f270 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:55.485693 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c14c2c-1691-4951-bf2c-6141ac1ac7cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:55.486359 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c14c2c-1691-4951-bf2c-6141ac1ac7cd service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:55.486642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c14c2c-1691-4951-bf2c-6141ac1ac7cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:55.487003 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c14c2c-1691-4951-bf2c-6141ac1ac7cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:55.491212 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-64c14c2c-1691-4951-bf2c-6141ac1ac7cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:55.579415 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c14c2c-1691-4951-bf2c-6141ac1ac7cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:55.581091 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c14c2c-1691-4951-bf2c-6141ac1ac7cd service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:55.595056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 975/1927] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:56.023588 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:56.299641 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6dfac9-273b-4cf6-9c25-d949d1c6c65a tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:56.302146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ca6dfac9-273b-4cf6-9c25-d949d1c6c65a tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:56.319866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6dfac9-273b-4cf6-9c25-d949d1c6c65a tempest-VolumeQuotaClassesTest-203857428 tempest-VolumeQuotaClassesTest-203857428-project-admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:56.325535 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74e400-6dca-483a-9baf-423f81440af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:56.327789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74e400-6dca-483a-9baf-423f81440af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:56.327789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74e400-6dca-483a-9baf-423f81440af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:56.327789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74e400-6dca-483a-9baf-423f81440af0 admin admin] PATH_INFO: `/v3/projects/fa5d6717df8147e88a9afaea2a5d1a30` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:56.329224 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74e400-6dca-483a-9baf-423f81440af0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa5d6717df8147e88a9afaea2a5d1a30)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:56.331636 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74e400-6dca-483a-9baf-423f81440af0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:56.499883 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ab74e400-6dca-483a-9baf-423f81440af0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa5d6717df8147e88a9afaea2a5d1a30', 'request_id': 'req-ab74e400-6dca-483a-9baf-423f81440af0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:56.527519 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ab74e400-6dca-483a-9baf-423f81440af0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa5d6717df8147e88a9afaea2a5d1a30', 'request_id': 'req-ab74e400-6dca-483a-9baf-423f81440af0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:56.535199 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ab74e400-6dca-483a-9baf-423f81440af0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa5d6717df8147e88a9afaea2a5d1a30', 'request_id': 'req-ab74e400-6dca-483a-9baf-423f81440af0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:56.669095 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ab74e400-6dca-483a-9baf-423f81440af0 admin admin] The token cache is being invalidate because project fa5d6717df8147e88a9afaea2a5d1a30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:56.669581 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ab74e400-6dca-483a-9baf-423f81440af0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:56.670849 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ab74e400-6dca-483a-9baf-423f81440af0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:56.672489 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 953/1928] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:56 2022] DELETE /identity/v3/projects/fa5d6717df8147e88a9afaea2a5d1a30 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:56.676714 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c14c2c-1691-4951-bf2c-6141ac1ac7cd service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:56.688780 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c14c2c-1691-4951-bf2c-6141ac1ac7cd service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:56.777652 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff3d1b-58d7-47a1-b568-712627171597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:56.783117 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff3d1b-58d7-47a1-b568-712627171597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:56.792155 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff3d1b-58d7-47a1-b568-712627171597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:56.792155 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff3d1b-58d7-47a1-b568-712627171597 admin admin] PATH_INFO: `/v3/users/276fed1aef864a3f954d4a17ad3ad28b` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:56.792155 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff3d1b-58d7-47a1-b568-712627171597 admin admin] RBAC: Authorizing `identity:delete_user(user_id=276fed1aef864a3f954d4a17ad3ad28b)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:56.792155 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff3d1b-58d7-47a1-b568-712627171597 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:56.981790 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74e400-6dca-483a-9baf-423f81440af0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:56.983210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ab74e400-6dca-483a-9baf-423f81440af0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:56.999917 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74e400-6dca-483a-9baf-423f81440af0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:57.005118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f1188-db3d-467a-86b1-b66e3acaf531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:57.006786 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f1188-db3d-467a-86b1-b66e3acaf531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:57.007142 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f1188-db3d-467a-86b1-b66e3acaf531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:57.007488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f1188-db3d-467a-86b1-b66e3acaf531 admin admin] PATH_INFO: `/v3/users/2b7afb10450c4c1b8b269e2cc16dab91` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:57.008970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f1188-db3d-467a-86b1-b66e3acaf531 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b7afb10450c4c1b8b269e2cc16dab91)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:57.011072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f1188-db3d-467a-86b1-b66e3acaf531 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:57.067594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-eeff3d1b-58d7-47a1-b568-712627171597 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '276fed1aef864a3f954d4a17ad3ad28b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:57.070099 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-eeff3d1b-58d7-47a1-b568-712627171597 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '276fed1aef864a3f954d4a17ad3ad28b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:57.083523 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-eeff3d1b-58d7-47a1-b568-712627171597 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '276fed1aef864a3f954d4a17ad3ad28b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:57.084798 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-eeff3d1b-58d7-47a1-b568-712627171597 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '276fed1aef864a3f954d4a17ad3ad28b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:57.175145 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-eeff3d1b-58d7-47a1-b568-712627171597 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '276fed1aef864a3f954d4a17ad3ad28b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:57.227132 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:57.229695 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 976/1929] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:56 2022] DELETE /identity/v3/users/276fed1aef864a3f954d4a17ad3ad28b => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:57.236913 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-802f1188-db3d-467a-86b1-b66e3acaf531 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b7afb10450c4c1b8b269e2cc16dab91'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:57.238257 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-802f1188-db3d-467a-86b1-b66e3acaf531 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b7afb10450c4c1b8b269e2cc16dab91'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:57.248420 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-802f1188-db3d-467a-86b1-b66e3acaf531 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b7afb10450c4c1b8b269e2cc16dab91'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:57.249478 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-802f1188-db3d-467a-86b1-b66e3acaf531 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b7afb10450c4c1b8b269e2cc16dab91'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:57.269873 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-802f1188-db3d-467a-86b1-b66e3acaf531 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b7afb10450c4c1b8b269e2cc16dab91'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:57.407426 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 954/1930] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:56 2022] DELETE /identity/v3/users/2b7afb10450c4c1b8b269e2cc16dab91 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:38:57.413186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c37a6c-a0b1-43c2-b1c9-29c701f2bc08 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:57.413947 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c37a6c-a0b1-43c2-b1c9-29c701f2bc08 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:57.414587 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c37a6c-a0b1-43c2-b1c9-29c701f2bc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:57.447493 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-27c37a6c-a0b1-43c2-b1c9-29c701f2bc08 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:57.448649 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-27c37a6c-a0b1-43c2-b1c9-29c701f2bc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:57.549580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 977/1931] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:57.585684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f1188-db3d-467a-86b1-b66e3acaf531 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:57.588300 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-802f1188-db3d-467a-86b1-b66e3acaf531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:57.684427 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f1188-db3d-467a-86b1-b66e3acaf531 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:57.690340 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94926b-a8d2-4115-96f5-bde2a0186e92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:57.691322 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94926b-a8d2-4115-96f5-bde2a0186e92 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:57.692160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94926b-a8d2-4115-96f5-bde2a0186e92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:57.692595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94926b-a8d2-4115-96f5-bde2a0186e92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:57.707097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a94926b-a8d2-4115-96f5-bde2a0186e92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:57.711063 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a94926b-a8d2-4115-96f5-bde2a0186e92 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:57.730441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 955/1932] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:57.879208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c37a6c-a0b1-43c2-b1c9-29c701f2bc08 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:57.880797 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-27c37a6c-a0b1-43c2-b1c9-29c701f2bc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:57.899273 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c37a6c-a0b1-43c2-b1c9-29c701f2bc08 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:57.915432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357943fc-6d3d-4686-923e-befbaba08928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:57.916522 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357943fc-6d3d-4686-923e-befbaba08928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:57.917096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357943fc-6d3d-4686-923e-befbaba08928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:57.917524 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357943fc-6d3d-4686-923e-befbaba08928 admin admin] PATH_INFO: `/v3/projects/f176de79f5a94ffca08cdac388a31a21` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:57.919105 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357943fc-6d3d-4686-923e-befbaba08928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f176de79f5a94ffca08cdac388a31a21)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:57.925855 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357943fc-6d3d-4686-923e-befbaba08928 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:58.003440 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:58.071077 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-357943fc-6d3d-4686-923e-befbaba08928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f176de79f5a94ffca08cdac388a31a21', 'request_id': 'req-357943fc-6d3d-4686-923e-befbaba08928'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:58.079846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-357943fc-6d3d-4686-923e-befbaba08928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f176de79f5a94ffca08cdac388a31a21', 'request_id': 'req-357943fc-6d3d-4686-923e-befbaba08928'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:58.796353 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-357943fc-6d3d-4686-923e-befbaba08928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f176de79f5a94ffca08cdac388a31a21', 'request_id': 'req-357943fc-6d3d-4686-923e-befbaba08928'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:58.796353 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-357943fc-6d3d-4686-923e-befbaba08928 admin admin] The token cache is being invalidate because project f176de79f5a94ffca08cdac388a31a21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:58.796353 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-357943fc-6d3d-4686-923e-befbaba08928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:58.796353 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-357943fc-6d3d-4686-923e-befbaba08928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:58.796353 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 978/1933] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:57 2022] DELETE /identity/v3/projects/f176de79f5a94ffca08cdac388a31a21 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:58.837242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d21288-7867-494f-b81c-30b857146c5e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:58.837242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d21288-7867-494f-b81c-30b857146c5e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:58.837242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d21288-7867-494f-b81c-30b857146c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:58.837242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-27d21288-7867-494f-b81c-30b857146c5e None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:58.837242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-27d21288-7867-494f-b81c-30b857146c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:58.837242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 956/1934] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:38:58.838993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5b272-3358-4ce4-bf8f-68077224cc32 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:58.838993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5b272-3358-4ce4-bf8f-68077224cc32 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:58.838993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5b272-3358-4ce4-bf8f-68077224cc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:58.838993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a0c5b272-3358-4ce4-bf8f-68077224cc32 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:58.838993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a0c5b272-3358-4ce4-bf8f-68077224cc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:58.838993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 979/1935] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:58.840403 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d21288-7867-494f-b81c-30b857146c5e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:58.840403 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d21288-7867-494f-b81c-30b857146c5e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:58.840403 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4949679-336a-4f4e-9f57-25fc97ff83de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:58.840403 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4949679-336a-4f4e-9f57-25fc97ff83de admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:58.841586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4949679-336a-4f4e-9f57-25fc97ff83de admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:58.841586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4949679-336a-4f4e-9f57-25fc97ff83de admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:58.841586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4949679-336a-4f4e-9f57-25fc97ff83de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:38:58.841586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4949679-336a-4f4e-9f57-25fc97ff83de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:58.841586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4949679-336a-4f4e-9f57-25fc97ff83de admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:58.841586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 957/1936] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:38:58.842371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5b272-3358-4ce4-bf8f-68077224cc32 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:58.842371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5b272-3358-4ce4-bf8f-68077224cc32 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:58.842371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ee1c1-9112-410d-bdd4-2e7e3d06fcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:58.842371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ee1c1-9112-410d-bdd4-2e7e3d06fcb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:58.843148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ee1c1-9112-410d-bdd4-2e7e3d06fcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:58.843148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ee1c1-9112-410d-bdd4-2e7e3d06fcb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:58.843148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ee1c1-9112-410d-bdd4-2e7e3d06fcb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:58.843148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ee1c1-9112-410d-bdd4-2e7e3d06fcb5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:58.871742 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 980/1937] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:58 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:38:58.874053 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4949679-336a-4f4e-9f57-25fc97ff83de admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:58.883341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4949679-336a-4f4e-9f57-25fc97ff83de admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:58.905934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ee1c1-9112-410d-bdd4-2e7e3d06fcb5 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:58.908018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4c3ee1c1-9112-410d-bdd4-2e7e3d06fcb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:58.982724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f91a32-00c1-470d-b542-4a62715c4a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:58.986975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:58.986975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f91a32-00c1-470d-b542-4a62715c4a5e admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:58.986975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f91a32-00c1-470d-b542-4a62715c4a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:58.986975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f91a32-00c1-470d-b542-4a62715c4a5e admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:58.988572 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f91a32-00c1-470d-b542-4a62715c4a5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:58.990847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f91a32-00c1-470d-b542-4a62715c4a5e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:59.016623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ee1c1-9112-410d-bdd4-2e7e3d06fcb5 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:59.024416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27506f37-a483-48c3-a21b-21d0dd8a6b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:59.038731 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27506f37-a483-48c3-a21b-21d0dd8a6b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:59.038731 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27506f37-a483-48c3-a21b-21d0dd8a6b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:59.038731 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27506f37-a483-48c3-a21b-21d0dd8a6b3d admin admin] PATH_INFO: `/v3/projects/a2ed08ea659e49f8826b87c2d72f605f` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:59.038731 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27506f37-a483-48c3-a21b-21d0dd8a6b3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2ed08ea659e49f8826b87c2d72f605f)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:59.038731 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27506f37-a483-48c3-a21b-21d0dd8a6b3d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:59.132498 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 958/1938] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:38:58 2022] POST /identity/v3/users => generated 291 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:38:59.133931 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f91a32-00c1-470d-b542-4a62715c4a5e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:59.157723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f91a32-00c1-470d-b542-4a62715c4a5e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:59.169390 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-27506f37-a483-48c3-a21b-21d0dd8a6b3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2ed08ea659e49f8826b87c2d72f605f', 'request_id': 'req-27506f37-a483-48c3-a21b-21d0dd8a6b3d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:59.205089 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-27506f37-a483-48c3-a21b-21d0dd8a6b3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2ed08ea659e49f8826b87c2d72f605f', 'request_id': 'req-27506f37-a483-48c3-a21b-21d0dd8a6b3d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:59.276255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6f3fd-1da8-4088-b698-21177a7e8803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:59.276961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6f3fd-1da8-4088-b698-21177a7e8803 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:59.277286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6f3fd-1da8-4088-b698-21177a7e8803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:59.277570 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6f3fd-1da8-4088-b698-21177a7e8803 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:59.278315 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6f3fd-1da8-4088-b698-21177a7e8803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:59.279934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6f3fd-1da8-4088-b698-21177a7e8803 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:59.297469 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 959/1939] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:59.299547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6f3fd-1da8-4088-b698-21177a7e8803 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:59.313478 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6f3fd-1da8-4088-b698-21177a7e8803 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:59.320554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0a257-2324-40da-8965-6dd4da35ad51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:59.321365 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0a257-2324-40da-8965-6dd4da35ad51 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:59.321716 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0a257-2324-40da-8965-6dd4da35ad51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:59.322076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0a257-2324-40da-8965-6dd4da35ad51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:59.322915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b0a257-2324-40da-8965-6dd4da35ad51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:59.325398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b0a257-2324-40da-8965-6dd4da35ad51 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:59.351950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-27506f37-a483-48c3-a21b-21d0dd8a6b3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2ed08ea659e49f8826b87c2d72f605f', 'request_id': 'req-27506f37-a483-48c3-a21b-21d0dd8a6b3d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:59.364848 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 960/1940] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:38:59.366489 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0a257-2324-40da-8965-6dd4da35ad51 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:59.367661 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-27506f37-a483-48c3-a21b-21d0dd8a6b3d admin admin] The token cache is being invalidate because project a2ed08ea659e49f8826b87c2d72f605f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:38:59.368891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-27506f37-a483-48c3-a21b-21d0dd8a6b3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:59.370126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-27506f37-a483-48c3-a21b-21d0dd8a6b3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:59.375117 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 981/1941] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:38:58 2022] DELETE /identity/v3/projects/a2ed08ea659e49f8826b87c2d72f605f => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:59.385324 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27506f37-a483-48c3-a21b-21d0dd8a6b3d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:59.386981 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-27506f37-a483-48c3-a21b-21d0dd8a6b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:59.387776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0a257-2324-40da-8965-6dd4da35ad51 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:59.394869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edb455-d428-4299-8514-f510e7ceab7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:59.396064 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edb455-d428-4299-8514-f510e7ceab7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:59.396064 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edb455-d428-4299-8514-f510e7ceab7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:59.396390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edb455-d428-4299-8514-f510e7ceab7b admin admin] PATH_INFO: `/v3/projects/62ac248ca98d4b59828a0acc450e299d/users/ad4bdd987a814c639dd020c6244ab06b/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:59.404388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27506f37-a483-48c3-a21b-21d0dd8a6b3d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:59.412805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ef201-7e99-4a70-a833-195523705a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:59.414663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ef201-7e99-4a70-a833-195523705a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:59.415193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ef201-7e99-4a70-a833-195523705a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:59.415660 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ef201-7e99-4a70-a833-195523705a67 admin admin] PATH_INFO: `/v3/users/e34d059b3a214b6794bac92c4d07d991` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:59.417619 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ef201-7e99-4a70-a833-195523705a67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e34d059b3a214b6794bac92c4d07d991)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:59.420501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ef201-7e99-4a70-a833-195523705a67 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:59.424186 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3edb455-d428-4299-8514-f510e7ceab7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=62ac248ca98d4b59828a0acc450e299d, user_id=ad4bdd987a814c639dd020c6244ab06b, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:59.438373 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3edb455-d428-4299-8514-f510e7ceab7b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:59.523549 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 961/1942] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:38:59 2022] PUT /identity/v3/projects/62ac248ca98d4b59828a0acc450e299d/users/ad4bdd987a814c639dd020c6244ab06b/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:59.525305 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edb455-d428-4299-8514-f510e7ceab7b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:59.527232 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c3edb455-d428-4299-8514-f510e7ceab7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:59.591836 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edb455-d428-4299-8514-f510e7ceab7b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:59.597197 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5891e0-1a52-443f-b1dc-6d27a9a1a979 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:59.601124 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5891e0-1a52-443f-b1dc-6d27a9a1a979 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:59.601124 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5891e0-1a52-443f-b1dc-6d27a9a1a979 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:59.601124 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5891e0-1a52-443f-b1dc-6d27a9a1a979 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:59.601124 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fb5891e0-1a52-443f-b1dc-6d27a9a1a979 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:59.674434 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5891e0-1a52-443f-b1dc-6d27a9a1a979 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:38:59.675925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5891e0-1a52-443f-b1dc-6d27a9a1a979 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:38:59.688204 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 962/1943] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:38:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:38:59.689145 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:38:59.691012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f571c-c10d-4301-a66e-a2d6ec7831b9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:59.691561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f571c-c10d-4301-a66e-a2d6ec7831b9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:59.692039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f571c-c10d-4301-a66e-a2d6ec7831b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:59.729641 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7c8f571c-c10d-4301-a66e-a2d6ec7831b9 None None] MFA Rules not processed for user `ad4bdd987a814c639dd020c6244ab06b`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:38:59.729641 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7c8f571c-c10d-4301-a66e-a2d6ec7831b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:59.738269 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a88ef201-7e99-4a70-a833-195523705a67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e34d059b3a214b6794bac92c4d07d991'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:59.739384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a88ef201-7e99-4a70-a833-195523705a67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e34d059b3a214b6794bac92c4d07d991'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:59.755172 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a88ef201-7e99-4a70-a833-195523705a67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e34d059b3a214b6794bac92c4d07d991'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:59.756234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a88ef201-7e99-4a70-a833-195523705a67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e34d059b3a214b6794bac92c4d07d991'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:59.816495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 963/1944] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:38:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:38:59.817760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f571c-c10d-4301-a66e-a2d6ec7831b9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:38:59.819503 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7c8f571c-c10d-4301-a66e-a2d6ec7831b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:38:59.820211 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a88ef201-7e99-4a70-a833-195523705a67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e34d059b3a214b6794bac92c4d07d991'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:38:59.857977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 982/1945] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:38:59 2022] DELETE /identity/v3/users/e34d059b3a214b6794bac92c4d07d991 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:38:59.910997 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f571c-c10d-4301-a66e-a2d6ec7831b9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:38:59.996576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b71c13-8bdc-47ae-bf7f-7d5b15249d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:38:59.997353 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b71c13-8bdc-47ae-bf7f-7d5b15249d41 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:38:59.997803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b71c13-8bdc-47ae-bf7f-7d5b15249d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:38:59.998091 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b71c13-8bdc-47ae-bf7f-7d5b15249d41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:38:59.998797 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b71c13-8bdc-47ae-bf7f-7d5b15249d41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:00.000948 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b71c13-8bdc-47ae-bf7f-7d5b15249d41 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:00.117578 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 964/1946] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:38:59 2022] POST /identity/v3/projects => generated 381 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:00.126087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ef201-7e99-4a70-a833-195523705a67 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:00.137559 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ef201-7e99-4a70-a833-195523705a67 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:00.213921 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159de91e-97c6-4981-858e-b3b068ea096b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:00.214762 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159de91e-97c6-4981-858e-b3b068ea096b admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:00.215156 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159de91e-97c6-4981-858e-b3b068ea096b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:00.215553 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159de91e-97c6-4981-858e-b3b068ea096b admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:00.216859 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159de91e-97c6-4981-858e-b3b068ea096b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:00.219277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159de91e-97c6-4981-858e-b3b068ea096b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:00.346064 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 983/1947] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:00 2022] POST /identity/v3/users => generated 297 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:00.350356 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b71c13-8bdc-47ae-bf7f-7d5b15249d41 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:00.360912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b71c13-8bdc-47ae-bf7f-7d5b15249d41 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:00.365587 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a2780-50e0-4c86-83ec-f2101ceca77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:00.366273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a2780-50e0-4c86-83ec-f2101ceca77c admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:00.366553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a2780-50e0-4c86-83ec-f2101ceca77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:00.366852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a2780-50e0-4c86-83ec-f2101ceca77c admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:00.367651 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a2780-50e0-4c86-83ec-f2101ceca77c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:00.369287 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a2780-50e0-4c86-83ec-f2101ceca77c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:00.380664 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 965/1948] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:00.385756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159de91e-97c6-4981-858e-b3b068ea096b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:00.397598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a2780-50e0-4c86-83ec-f2101ceca77c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:00.398727 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159de91e-97c6-4981-858e-b3b068ea096b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:00.399405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-978a2780-50e0-4c86-83ec-f2101ceca77c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:00.403487 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606da070-ab05-4907-9a14-4d25b94261cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:00.404554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606da070-ab05-4907-9a14-4d25b94261cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:00.404838 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606da070-ab05-4907-9a14-4d25b94261cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:00.405328 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606da070-ab05-4907-9a14-4d25b94261cc admin admin] PATH_INFO: `/v3/projects/f078798a21d7423ebcd371fe37adb8ca/users/83d08d82c20046d693391c8b1a0701cb/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:00.414886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a2780-50e0-4c86-83ec-f2101ceca77c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:00.421061 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6229ec16-ebc5-4e80-a31b-7c5d3d815304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:00.422569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229ec16-ebc5-4e80-a31b-7c5d3d815304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:00.423704 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229ec16-ebc5-4e80-a31b-7c5d3d815304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:00.424165 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229ec16-ebc5-4e80-a31b-7c5d3d815304 admin admin] PATH_INFO: `/v3/projects/2c204eebfd474bd1b5bff4e1da4cae26` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:00.426246 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6229ec16-ebc5-4e80-a31b-7c5d3d815304 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c204eebfd474bd1b5bff4e1da4cae26)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:00.427503 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606da070-ab05-4907-9a14-4d25b94261cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f078798a21d7423ebcd371fe37adb8ca, user_id=83d08d82c20046d693391c8b1a0701cb, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:00.428908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6229ec16-ebc5-4e80-a31b-7c5d3d815304 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:00.432246 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606da070-ab05-4907-9a14-4d25b94261cc admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:00.488796 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:00.591938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 984/1949] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:00 2022] PUT /identity/v3/projects/f078798a21d7423ebcd371fe37adb8ca/users/83d08d82c20046d693391c8b1a0701cb/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:00.597931 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606da070-ab05-4907-9a14-4d25b94261cc admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:00.661708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6229ec16-ebc5-4e80-a31b-7c5d3d815304 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c204eebfd474bd1b5bff4e1da4cae26', 'request_id': 'req-6229ec16-ebc5-4e80-a31b-7c5d3d815304'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:00.667995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606da070-ab05-4907-9a14-4d25b94261cc admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:00.670500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6229ec16-ebc5-4e80-a31b-7c5d3d815304 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c204eebfd474bd1b5bff4e1da4cae26', 'request_id': 'req-6229ec16-ebc5-4e80-a31b-7c5d3d815304'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:00.679379 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6229ec16-ebc5-4e80-a31b-7c5d3d815304 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c204eebfd474bd1b5bff4e1da4cae26', 'request_id': 'req-6229ec16-ebc5-4e80-a31b-7c5d3d815304'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:00.731113 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfc7de-4e5d-4cd6-932d-b664bd167f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:00.731757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfc7de-4e5d-4cd6-932d-b664bd167f54 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:00.732034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfc7de-4e5d-4cd6-932d-b664bd167f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:00.732321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfc7de-4e5d-4cd6-932d-b664bd167f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:00.733135 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfc7de-4e5d-4cd6-932d-b664bd167f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:00.734742 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfc7de-4e5d-4cd6-932d-b664bd167f54 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:00.745562 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 985/1950] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:39:00.749756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfc7de-4e5d-4cd6-932d-b664bd167f54 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:00.761786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfc7de-4e5d-4cd6-932d-b664bd167f54 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:00.762936 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6229ec16-ebc5-4e80-a31b-7c5d3d815304 admin admin] The token cache is being invalidate because project 2c204eebfd474bd1b5bff4e1da4cae26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:00.763221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6229ec16-ebc5-4e80-a31b-7c5d3d815304 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:00.763929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6229ec16-ebc5-4e80-a31b-7c5d3d815304 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:00.765775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 966/1951] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:00 2022] DELETE /identity/v3/projects/2c204eebfd474bd1b5bff4e1da4cae26 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:00.769959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111130b-cf57-4b10-9bfd-af8b90374c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:00.770675 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c4d18-3b7e-4ae0-8a8e-3bf6f2232f5b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:00.770878 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111130b-cf57-4b10-9bfd-af8b90374c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:00.771127 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111130b-cf57-4b10-9bfd-af8b90374c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:00.771312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c4d18-3b7e-4ae0-8a8e-3bf6f2232f5b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:00.771475 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111130b-cf57-4b10-9bfd-af8b90374c2f admin admin] PATH_INFO: `/v3/projects/f078798a21d7423ebcd371fe37adb8ca/users/83d08d82c20046d693391c8b1a0701cb/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:00.771631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c4d18-3b7e-4ae0-8a8e-3bf6f2232f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:00.773474 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111130b-cf57-4b10-9bfd-af8b90374c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f078798a21d7423ebcd371fe37adb8ca, user_id=83d08d82c20046d693391c8b1a0701cb, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:00.776158 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111130b-cf57-4b10-9bfd-af8b90374c2f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:00.793486 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 986/1952] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:00 2022] PUT /identity/v3/projects/f078798a21d7423ebcd371fe37adb8ca/users/83d08d82c20046d693391c8b1a0701cb/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:00.795074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1f3c4d18-3b7e-4ae0-8a8e-3bf6f2232f5b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:00.796398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1f3c4d18-3b7e-4ae0-8a8e-3bf6f2232f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:00.806823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbec5e-de3d-4f5a-aae2-81bdb373aedd None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:00.807535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbec5e-de3d-4f5a-aae2-81bdb373aedd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:00.807535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbec5e-de3d-4f5a-aae2-81bdb373aedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:00.839369 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-39bbec5e-de3d-4f5a-aae2-81bdb373aedd None None] MFA Rules not processed for user `83d08d82c20046d693391c8b1a0701cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:00.840638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-39bbec5e-de3d-4f5a-aae2-81bdb373aedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:00.876910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 967/1953] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:00.886031 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9aecc6-ffcc-469e-8c98-9080cf93a222 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:00.886412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9aecc6-ffcc-469e-8c98-9080cf93a222 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:00.886663 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9aecc6-ffcc-469e-8c98-9080cf93a222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:00.907679 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9e9aecc6-ffcc-469e-8c98-9080cf93a222 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:00.908037 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9e9aecc6-ffcc-469e-8c98-9080cf93a222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:00.919786 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 968/1954] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:00.924412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9aecc6-ffcc-469e-8c98-9080cf93a222 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:00.936077 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9aecc6-ffcc-469e-8c98-9080cf93a222 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:00.938282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 987/1955] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:00.943167 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98499d-6b11-467c-9f7e-770e291d1534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:00.944889 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98499d-6b11-467c-9f7e-770e291d1534 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:00.945236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98499d-6b11-467c-9f7e-770e291d1534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:00.945599 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98499d-6b11-467c-9f7e-770e291d1534 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:00.946437 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98499d-6b11-467c-9f7e-770e291d1534 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:39:00.946722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:00.946838 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98499d-6b11-467c-9f7e-770e291d1534 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:00.948686 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98499d-6b11-467c-9f7e-770e291d1534 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:00.957672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbec5e-de3d-4f5a-aae2-81bdb373aedd None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:00.960818 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-39bbec5e-de3d-4f5a-aae2-81bdb373aedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:00.973648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 969/1956] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:39:00.977978 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98499d-6b11-467c-9f7e-770e291d1534 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:00.980709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6b98499d-6b11-467c-9f7e-770e291d1534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:00.994322 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98499d-6b11-467c-9f7e-770e291d1534 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:00.999304 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674242fc-42e6-4784-b443-918da3a1755e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:01.000381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674242fc-42e6-4784-b443-918da3a1755e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:01.000659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674242fc-42e6-4784-b443-918da3a1755e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:01.000958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674242fc-42e6-4784-b443-918da3a1755e admin admin] PATH_INFO: `/v3/users/644a0203dbaa470690cd84862b2d891f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:01.004273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674242fc-42e6-4784-b443-918da3a1755e admin admin] RBAC: Authorizing `identity:delete_user(user_id=644a0203dbaa470690cd84862b2d891f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:01.006438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674242fc-42e6-4784-b443-918da3a1755e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:01.030823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbec5e-de3d-4f5a-aae2-81bdb373aedd None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:01.035279 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc8c01-786e-4c04-9830-6c34341949e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:01.035958 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc8c01-786e-4c04-9830-6c34341949e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:01.036233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc8c01-786e-4c04-9830-6c34341949e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:01.036555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc8c01-786e-4c04-9830-6c34341949e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:01.046753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc8c01-786e-4c04-9830-6c34341949e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:01.048196 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc8c01-786e-4c04-9830-6c34341949e3 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:01.059123 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 988/1957] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:00 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:01.059950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc8c01-786e-4c04-9830-6c34341949e3 service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:01.073731 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc8c01-786e-4c04-9830-6c34341949e3 service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:01.078880 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587560e9-c0e8-4985-8d53-303f5eed825c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:01.080045 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587560e9-c0e8-4985-8d53-303f5eed825c admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:01.080306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587560e9-c0e8-4985-8d53-303f5eed825c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:01.080833 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587560e9-c0e8-4985-8d53-303f5eed825c admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:01.082069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587560e9-c0e8-4985-8d53-303f5eed825c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:01.084720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587560e9-c0e8-4985-8d53-303f5eed825c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:01.199691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 989/1958] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:01 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:01.200622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587560e9-c0e8-4985-8d53-303f5eed825c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:01.260106 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587560e9-c0e8-4985-8d53-303f5eed825c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:01.264840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3ab75-2a89-41cf-9e15-26cfa94d814b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:01.264840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3ab75-2a89-41cf-9e15-26cfa94d814b service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:01.264840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3ab75-2a89-41cf-9e15-26cfa94d814b service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:01.264840 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3ab75-2a89-41cf-9e15-26cfa94d814b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:01.267048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1ec3ab75-2a89-41cf-9e15-26cfa94d814b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:01.287021 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-674242fc-42e6-4784-b443-918da3a1755e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '644a0203dbaa470690cd84862b2d891f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:01.287849 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-674242fc-42e6-4784-b443-918da3a1755e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '644a0203dbaa470690cd84862b2d891f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:01.296363 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-674242fc-42e6-4784-b443-918da3a1755e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '644a0203dbaa470690cd84862b2d891f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:01.297223 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-674242fc-42e6-4784-b443-918da3a1755e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '644a0203dbaa470690cd84862b2d891f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:01.318082 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-674242fc-42e6-4784-b443-918da3a1755e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '644a0203dbaa470690cd84862b2d891f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:01.338831 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3ab75-2a89-41cf-9e15-26cfa94d814b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:01.340309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3ab75-2a89-41cf-9e15-26cfa94d814b service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:01.349650 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 990/1959] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:01.350485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3ab75-2a89-41cf-9e15-26cfa94d814b service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:01.395243 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 970/1960] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:00 2022] DELETE /identity/v3/users/644a0203dbaa470690cd84862b2d891f => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:01.401319 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835c19c-a4a5-44e2-9e94-676521bfbedd None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:01.401919 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835c19c-a4a5-44e2-9e94-676521bfbedd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:01.402205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835c19c-a4a5-44e2-9e94-676521bfbedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:01.429224 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3835c19c-a4a5-44e2-9e94-676521bfbedd None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:01.431260 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3835c19c-a4a5-44e2-9e94-676521bfbedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:01.437704 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3ab75-2a89-41cf-9e15-26cfa94d814b service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:01.517403 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd831046-6052-4693-bba0-375cb9cd59c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:01.518125 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd831046-6052-4693-bba0-375cb9cd59c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:01.518493 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd831046-6052-4693-bba0-375cb9cd59c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:01.518796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd831046-6052-4693-bba0-375cb9cd59c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:01.519567 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd831046-6052-4693-bba0-375cb9cd59c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:01.521348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 971/1961] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:01.522416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd831046-6052-4693-bba0-375cb9cd59c2 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:01.531535 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835c19c-a4a5-44e2-9e94-676521bfbedd None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:01.533342 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3835c19c-a4a5-44e2-9e94-676521bfbedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:01.601502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835c19c-a4a5-44e2-9e94-676521bfbedd None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:01.617736 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc758f9d-889b-435b-a873-108f5bb738b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:01.617736 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc758f9d-889b-435b-a873-108f5bb738b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:01.618062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc758f9d-889b-435b-a873-108f5bb738b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:01.618358 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc758f9d-889b-435b-a873-108f5bb738b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:01.623876 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 991/1962] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:01 2022] POST /identity/v3/users => generated 290 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:01.624750 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:01.628092 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd831046-6052-4693-bba0-375cb9cd59c2 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:01.631502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-dd831046-6052-4693-bba0-375cb9cd59c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:01.635912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc758f9d-889b-435b-a873-108f5bb738b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:01.637734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc758f9d-889b-435b-a873-108f5bb738b9 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:01.648892 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd831046-6052-4693-bba0-375cb9cd59c2 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:01.651976 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 972/1963] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:39:01.654763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b49e4-f0c3-4aa1-83f1-2652d1a900f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:01.655514 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b49e4-f0c3-4aa1-83f1-2652d1a900f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:01.655795 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b49e4-f0c3-4aa1-83f1-2652d1a900f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:01.656075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b49e4-f0c3-4aa1-83f1-2652d1a900f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:01.656807 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b49e4-f0c3-4aa1-83f1-2652d1a900f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:01.658683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b49e4-f0c3-4aa1-83f1-2652d1a900f4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:01.675242 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 992/1964] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:39:01.676976 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc758f9d-889b-435b-a873-108f5bb738b9 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:01.690938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc758f9d-889b-435b-a873-108f5bb738b9 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:01.697976 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1cd57-eef3-46a3-99f9-9c7d07788d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:01.699310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1cd57-eef3-46a3-99f9-9c7d07788d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:01.699742 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1cd57-eef3-46a3-99f9-9c7d07788d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:01.700133 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1cd57-eef3-46a3-99f9-9c7d07788d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:01.701193 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe1cd57-eef3-46a3-99f9-9c7d07788d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:01.703030 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe1cd57-eef3-46a3-99f9-9c7d07788d4d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:01.718417 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 973/1965] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:01.727285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b49e4-f0c3-4aa1-83f1-2652d1a900f4 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:01.743295 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b49e4-f0c3-4aa1-83f1-2652d1a900f4 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:01.749491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b9823-19b9-4366-b2fe-962aa23dc3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:01.750503 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b9823-19b9-4366-b2fe-962aa23dc3a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:01.750791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b9823-19b9-4366-b2fe-962aa23dc3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:01.751037 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b9823-19b9-4366-b2fe-962aa23dc3a9 admin admin] PATH_INFO: `/v3/projects/c8d8f5253c9241b0a3ca4e79db14f7ed/users/787c95aa412847eaa09ecf735829dcaa/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:01.769071 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b9823-19b9-4366-b2fe-962aa23dc3a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8d8f5253c9241b0a3ca4e79db14f7ed, user_id=787c95aa412847eaa09ecf735829dcaa, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:01.771813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b9823-19b9-4366-b2fe-962aa23dc3a9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:01.890488 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:01.901718 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 993/1966] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:01 2022] PUT /identity/v3/projects/c8d8f5253c9241b0a3ca4e79db14f7ed/users/787c95aa412847eaa09ecf735829dcaa/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:01.912759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3accfd-77c3-425b-a4ef-8edda1179914 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:01.913082 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3accfd-77c3-425b-a4ef-8edda1179914 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:01.913364 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3accfd-77c3-425b-a4ef-8edda1179914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:01.948222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-cd3accfd-77c3-425b-a4ef-8edda1179914 None None] MFA Rules not processed for user `787c95aa412847eaa09ecf735829dcaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:01.949932 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cd3accfd-77c3-425b-a4ef-8edda1179914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:02.031451 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 974/1967] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:02.032601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b9823-19b9-4366-b2fe-962aa23dc3a9 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:02.098336 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b9823-19b9-4366-b2fe-962aa23dc3a9 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:02.104697 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e9ea9-8dff-4f69-99a2-6252a2367a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:02.105450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e9ea9-8dff-4f69-99a2-6252a2367a31 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:02.105732 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e9ea9-8dff-4f69-99a2-6252a2367a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:02.106525 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e9ea9-8dff-4f69-99a2-6252a2367a31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:02.106770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e9ea9-8dff-4f69-99a2-6252a2367a31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:02.108717 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e9ea9-8dff-4f69-99a2-6252a2367a31 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:02.255556 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3accfd-77c3-425b-a4ef-8edda1179914 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:02.257044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cd3accfd-77c3-425b-a4ef-8edda1179914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:02.261461 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 994/1968] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:02 2022] POST /identity/v3/projects => generated 379 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:02.262012 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e9ea9-8dff-4f69-99a2-6252a2367a31 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:02.289414 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3accfd-77c3-425b-a4ef-8edda1179914 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:02.336796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e9ea9-8dff-4f69-99a2-6252a2367a31 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:02.341692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2b641-4729-431b-beff-78412080aeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:02.342380 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2b641-4729-431b-beff-78412080aeca admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:02.342660 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2b641-4729-431b-beff-78412080aeca admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:02.342939 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2b641-4729-431b-beff-78412080aeca admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:02.343636 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2b641-4729-431b-beff-78412080aeca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:02.345629 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2b641-4729-431b-beff-78412080aeca admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:02.357312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b359be5-2a26-4580-9b24-c3b4b48e5fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:02.358257 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b359be5-2a26-4580-9b24-c3b4b48e5fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:02.358534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b359be5-2a26-4580-9b24-c3b4b48e5fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:02.358856 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b359be5-2a26-4580-9b24-c3b4b48e5fc2 admin admin] PATH_INFO: `/v3/projects/6977726dec1740d48a80da6bdf81d9b4` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:02.360074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b359be5-2a26-4580-9b24-c3b4b48e5fc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6977726dec1740d48a80da6bdf81d9b4)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:02.362238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b359be5-2a26-4580-9b24-c3b4b48e5fc2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:02.387699 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:02.450024 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 995/1969] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:02 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:02.454052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2b641-4729-431b-beff-78412080aeca admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:02.465466 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2b641-4729-431b-beff-78412080aeca admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:02.470327 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e355b-7a1d-43d5-8b11-4a62bc0c6619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:02.470999 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e355b-7a1d-43d5-8b11-4a62bc0c6619 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:02.471288 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e355b-7a1d-43d5-8b11-4a62bc0c6619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:02.471557 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e355b-7a1d-43d5-8b11-4a62bc0c6619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:02.472289 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e355b-7a1d-43d5-8b11-4a62bc0c6619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:02.473945 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e355b-7a1d-43d5-8b11-4a62bc0c6619 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:02.484603 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 996/1970] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:39:02.488720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e355b-7a1d-43d5-8b11-4a62bc0c6619 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:02.498194 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b359be5-2a26-4580-9b24-c3b4b48e5fc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6977726dec1740d48a80da6bdf81d9b4', 'request_id': 'req-2b359be5-2a26-4580-9b24-c3b4b48e5fc2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.499139 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e355b-7a1d-43d5-8b11-4a62bc0c6619 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:02.504880 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b359be5-2a26-4580-9b24-c3b4b48e5fc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6977726dec1740d48a80da6bdf81d9b4', 'request_id': 'req-2b359be5-2a26-4580-9b24-c3b4b48e5fc2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.514444 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b359be5-2a26-4580-9b24-c3b4b48e5fc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6977726dec1740d48a80da6bdf81d9b4', 'request_id': 'req-2b359be5-2a26-4580-9b24-c3b4b48e5fc2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.559805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a2738-a1a0-4be2-b3d3-62b727f98107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:02.560469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a2738-a1a0-4be2-b3d3-62b727f98107 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:02.560748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a2738-a1a0-4be2-b3d3-62b727f98107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:02.561065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a2738-a1a0-4be2-b3d3-62b727f98107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:02.561832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a2738-a1a0-4be2-b3d3-62b727f98107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:02.563413 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a2738-a1a0-4be2-b3d3-62b727f98107 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:02.574296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 997/1971] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:39:02.578717 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a2738-a1a0-4be2-b3d3-62b727f98107 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:02.581136 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b359be5-2a26-4580-9b24-c3b4b48e5fc2 admin admin] The token cache is being invalidate because project 6977726dec1740d48a80da6bdf81d9b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:02.586592 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b359be5-2a26-4580-9b24-c3b4b48e5fc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.586592 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b359be5-2a26-4580-9b24-c3b4b48e5fc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.586592 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 975/1972] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:02 2022] DELETE /identity/v3/projects/6977726dec1740d48a80da6bdf81d9b4 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:39:02.588653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:02.589371 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b359be5-2a26-4580-9b24-c3b4b48e5fc2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:02.591066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2b359be5-2a26-4580-9b24-c3b4b48e5fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:02.594140 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a2738-a1a0-4be2-b3d3-62b727f98107 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:02.601789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f213ca-9ad0-46c2-a899-5225fc42d49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:02.604824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f213ca-9ad0-46c2-a899-5225fc42d49d admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:02.604824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f213ca-9ad0-46c2-a899-5225fc42d49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:02.604824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f213ca-9ad0-46c2-a899-5225fc42d49d admin admin] PATH_INFO: `/v3/projects/365dc71d274044d1985699f6501c7099/users/8666735682d648d1958a7db76bb98672/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:02.605488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b359be5-2a26-4580-9b24-c3b4b48e5fc2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:02.612801 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2b9b7-bfc9-49e2-bc4f-755e21ebb418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:02.613760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2b9b7-bfc9-49e2-bc4f-755e21ebb418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:02.620984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2b9b7-bfc9-49e2-bc4f-755e21ebb418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:02.620984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2b9b7-bfc9-49e2-bc4f-755e21ebb418 admin admin] PATH_INFO: `/v3/users/339cdb82fac5405695ce89f6354d00e6` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:02.620984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e2b9b7-bfc9-49e2-bc4f-755e21ebb418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=339cdb82fac5405695ce89f6354d00e6)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:02.620984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e2b9b7-bfc9-49e2-bc4f-755e21ebb418 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:02.621409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f213ca-9ad0-46c2-a899-5225fc42d49d admin admin] RBAC: Authorizing `identity:create_grant(project_id=365dc71d274044d1985699f6501c7099, user_id=8666735682d648d1958a7db76bb98672, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:02.621953 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f213ca-9ad0-46c2-a899-5225fc42d49d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:02.678477 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 998/1973] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:02 2022] PUT /identity/v3/projects/365dc71d274044d1985699f6501c7099/users/8666735682d648d1958a7db76bb98672/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:39:02.685284 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c979797-ddca-47e6-a134-723fd0ccf4ec None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:02.685569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c979797-ddca-47e6-a134-723fd0ccf4ec None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:02.685842 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c979797-ddca-47e6-a134-723fd0ccf4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:02.716092 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1c979797-ddca-47e6-a134-723fd0ccf4ec None None] MFA Rules not processed for user `8666735682d648d1958a7db76bb98672`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:02.717290 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1c979797-ddca-47e6-a134-723fd0ccf4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:02.794041 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-98e2b9b7-bfc9-49e2-bc4f-755e21ebb418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '339cdb82fac5405695ce89f6354d00e6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.794822 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-98e2b9b7-bfc9-49e2-bc4f-755e21ebb418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '339cdb82fac5405695ce89f6354d00e6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.804093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 999/1974] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:02.804316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-98e2b9b7-bfc9-49e2-bc4f-755e21ebb418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '339cdb82fac5405695ce89f6354d00e6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.804706 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-98e2b9b7-bfc9-49e2-bc4f-755e21ebb418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '339cdb82fac5405695ce89f6354d00e6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.805250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c979797-ddca-47e6-a134-723fd0ccf4ec None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:02.806570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1c979797-ddca-47e6-a134-723fd0ccf4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:02.817102 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-98e2b9b7-bfc9-49e2-bc4f-755e21ebb418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '339cdb82fac5405695ce89f6354d00e6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:02.838974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:02.883838 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c979797-ddca-47e6-a134-723fd0ccf4ec None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:02.891409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409a2cb-fcc2-40d5-836f-dd3b8bcea045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:02.892575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409a2cb-fcc2-40d5-836f-dd3b8bcea045 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:02.892940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409a2cb-fcc2-40d5-836f-dd3b8bcea045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:02.893773 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409a2cb-fcc2-40d5-836f-dd3b8bcea045 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:02.895126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409a2cb-fcc2-40d5-836f-dd3b8bcea045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:02.896222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 976/1975] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:02 2022] DELETE /identity/v3/users/339cdb82fac5405695ce89f6354d00e6 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:39:02.898901 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409a2cb-fcc2-40d5-836f-dd3b8bcea045 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:02.979261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1000/1976] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:02 2022] POST /identity/v3/projects => generated 379 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:02.982862 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2b9b7-bfc9-49e2-bc4f-755e21ebb418 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:02.984614 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-98e2b9b7-bfc9-49e2-bc4f-755e21ebb418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:03.009765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:03.065230 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2b9b7-bfc9-49e2-bc4f-755e21ebb418 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:03.127880 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e9332-ab75-42e6-83ac-202afde861fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:03.128617 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e9332-ab75-42e6-83ac-202afde861fb admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:03.129174 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e9332-ab75-42e6-83ac-202afde861fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:03.129174 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e9332-ab75-42e6-83ac-202afde861fb admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:03.129920 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e9332-ab75-42e6-83ac-202afde861fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:03.131837 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e9332-ab75-42e6-83ac-202afde861fb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:03.255795 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 977/1977] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:02 2022] POST /identity/v3/users => generated 296 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:03.260213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409a2cb-fcc2-40d5-836f-dd3b8bcea045 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:03.273068 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409a2cb-fcc2-40d5-836f-dd3b8bcea045 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:03.278842 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376c5dc-d964-42c6-8d4b-e2e93f883e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:03.279987 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376c5dc-d964-42c6-8d4b-e2e93f883e94 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:03.280529 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376c5dc-d964-42c6-8d4b-e2e93f883e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:03.280832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376c5dc-d964-42c6-8d4b-e2e93f883e94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:03.282125 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376c5dc-d964-42c6-8d4b-e2e93f883e94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:03.284518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376c5dc-d964-42c6-8d4b-e2e93f883e94 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:03.303765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1001/1978] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:03.308039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e9332-ab75-42e6-83ac-202afde861fb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:03.323406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e9332-ab75-42e6-83ac-202afde861fb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:03.328264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:03.332485 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb6e98-c2cd-47b2-9739-297eaa3289f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:03.333435 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb6e98-c2cd-47b2-9739-297eaa3289f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:03.333716 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb6e98-c2cd-47b2-9739-297eaa3289f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:03.333992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb6e98-c2cd-47b2-9739-297eaa3289f1 admin admin] PATH_INFO: `/v3/projects/ac240114eedc4eae8ff8d99267f05601/users/cb5a3abc3797419e82eb045cf0487d52/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:03.362164 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376c5dc-d964-42c6-8d4b-e2e93f883e94 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:03.364861 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d376c5dc-d964-42c6-8d4b-e2e93f883e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:03.365239 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb6e98-c2cd-47b2-9739-297eaa3289f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac240114eedc4eae8ff8d99267f05601, user_id=cb5a3abc3797419e82eb045cf0487d52, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:03.370590 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb6e98-c2cd-47b2-9739-297eaa3289f1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:03.384428 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376c5dc-d964-42c6-8d4b-e2e93f883e94 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:03.390635 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15218180-d92f-441d-9050-f56b66e64b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:03.391746 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15218180-d92f-441d-9050-f56b66e64b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:03.397220 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15218180-d92f-441d-9050-f56b66e64b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:03.397220 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15218180-d92f-441d-9050-f56b66e64b4c admin admin] PATH_INFO: `/v3/projects/96f5891b5086440ca658e684bcc25063` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:03.397220 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15218180-d92f-441d-9050-f56b66e64b4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=96f5891b5086440ca658e684bcc25063)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:03.397220 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15218180-d92f-441d-9050-f56b66e64b4c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:03.489736 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 978/1979] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:03 2022] PUT /identity/v3/projects/ac240114eedc4eae8ff8d99267f05601/users/cb5a3abc3797419e82eb045cf0487d52/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:03.494672 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb6e98-c2cd-47b2-9739-297eaa3289f1 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:03.533030 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-15218180-d92f-441d-9050-f56b66e64b4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96f5891b5086440ca658e684bcc25063', 'request_id': 'req-15218180-d92f-441d-9050-f56b66e64b4c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:03.542173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-15218180-d92f-441d-9050-f56b66e64b4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96f5891b5086440ca658e684bcc25063', 'request_id': 'req-15218180-d92f-441d-9050-f56b66e64b4c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:03.583926 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb6e98-c2cd-47b2-9739-297eaa3289f1 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:03.598716 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-15218180-d92f-441d-9050-f56b66e64b4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96f5891b5086440ca658e684bcc25063', 'request_id': 'req-15218180-d92f-441d-9050-f56b66e64b4c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:03.611158 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:03.617749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-15218180-d92f-441d-9050-f56b66e64b4c admin admin] The token cache is being invalidate because project 96f5891b5086440ca658e684bcc25063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:03.618043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-15218180-d92f-441d-9050-f56b66e64b4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:03.619226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-15218180-d92f-441d-9050-f56b66e64b4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:03.620392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1002/1980] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:03 2022] DELETE /identity/v3/projects/96f5891b5086440ca658e684bcc25063 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:03.625910 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450a985-3fa4-4863-a79b-da19e87079bd None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:03.626202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450a985-3fa4-4863-a79b-da19e87079bd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:03.626467 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450a985-3fa4-4863-a79b-da19e87079bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:03.646330 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5807d0b-4fc1-4e4f-b55f-bc69c9274d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:03.647558 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a450a985-3fa4-4863-a79b-da19e87079bd None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:03.647702 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5807d0b-4fc1-4e4f-b55f-bc69c9274d50 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:03.647875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5807d0b-4fc1-4e4f-b55f-bc69c9274d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:03.648153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5807d0b-4fc1-4e4f-b55f-bc69c9274d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:03.648787 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a450a985-3fa4-4863-a79b-da19e87079bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:03.648906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5807d0b-4fc1-4e4f-b55f-bc69c9274d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:03.650827 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5807d0b-4fc1-4e4f-b55f-bc69c9274d50 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:03.664869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 979/1981] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:39:03.669390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5807d0b-4fc1-4e4f-b55f-bc69c9274d50 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:03.672374 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1003/1982] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:03.672521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b5807d0b-4fc1-4e4f-b55f-bc69c9274d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:03.678126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec2487-5e07-4667-8efa-4af8343a250f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:03.678414 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec2487-5e07-4667-8efa-4af8343a250f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:03.678696 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec2487-5e07-4667-8efa-4af8343a250f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:03.688940 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5807d0b-4fc1-4e4f-b55f-bc69c9274d50 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:03.695033 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1de7c-050a-4069-9d11-3fd1c28eb886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:03.695845 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1de7c-050a-4069-9d11-3fd1c28eb886 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:03.696149 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1de7c-050a-4069-9d11-3fd1c28eb886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:03.696405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1de7c-050a-4069-9d11-3fd1c28eb886 admin admin] PATH_INFO: `/v3/projects/ac240114eedc4eae8ff8d99267f05601/users/cb5a3abc3797419e82eb045cf0487d52/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:03.698618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c1de7c-050a-4069-9d11-3fd1c28eb886 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac240114eedc4eae8ff8d99267f05601, user_id=cb5a3abc3797419e82eb045cf0487d52, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:03.701343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-22ec2487-5e07-4667-8efa-4af8343a250f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:03.701880 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c1de7c-050a-4069-9d11-3fd1c28eb886 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:03.702602 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-22ec2487-5e07-4667-8efa-4af8343a250f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:03.712479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1004/1983] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:03.716435 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 980/1984] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:03 2022] PUT /identity/v3/projects/ac240114eedc4eae8ff8d99267f05601/users/cb5a3abc3797419e82eb045cf0487d52/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:39:03.718096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec2487-5e07-4667-8efa-4af8343a250f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:03.734474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22f787-8306-44a1-a410-489e19ea0de8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:03.734957 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22f787-8306-44a1-a410-489e19ea0de8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:03.735397 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22f787-8306-44a1-a410-489e19ea0de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:03.743161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec2487-5e07-4667-8efa-4af8343a250f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:03.778358 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9f22f787-8306-44a1-a410-489e19ea0de8 None None] MFA Rules not processed for user `cb5a3abc3797419e82eb045cf0487d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:03.779689 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9f22f787-8306-44a1-a410-489e19ea0de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:03.847218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fe722-7be7-4e5c-8c35-7e3e2e215706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:03.847875 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fe722-7be7-4e5c-8c35-7e3e2e215706 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:03.848153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fe722-7be7-4e5c-8c35-7e3e2e215706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:03.848427 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fe722-7be7-4e5c-8c35-7e3e2e215706 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:03.849197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fe722-7be7-4e5c-8c35-7e3e2e215706 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:39:03.849505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fe722-7be7-4e5c-8c35-7e3e2e215706 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:03.852213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fe722-7be7-4e5c-8c35-7e3e2e215706 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:03.878665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1005/1985] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:39:03.879592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fe722-7be7-4e5c-8c35-7e3e2e215706 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:03.886061 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:03.893221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 981/1986] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:03 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:03.902489 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fe722-7be7-4e5c-8c35-7e3e2e215706 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:03.909846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1c375-8a0a-45e4-b62f-7f9b09a1b2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:03.910693 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1c375-8a0a-45e4-b62f-7f9b09a1b2d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:03.910986 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1c375-8a0a-45e4-b62f-7f9b09a1b2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:03.911267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1c375-8a0a-45e4-b62f-7f9b09a1b2d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:03.911984 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1c375-8a0a-45e4-b62f-7f9b09a1b2d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:03.914356 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1c375-8a0a-45e4-b62f-7f9b09a1b2d3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:03.934366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22f787-8306-44a1-a410-489e19ea0de8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:03.936175 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9f22f787-8306-44a1-a410-489e19ea0de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:04.009425 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22f787-8306-44a1-a410-489e19ea0de8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:04.013747 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80896bad-c89c-4314-85dc-1b735eaff9e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:04.019770 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80896bad-c89c-4314-85dc-1b735eaff9e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:04.019770 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80896bad-c89c-4314-85dc-1b735eaff9e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:04.019770 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80896bad-c89c-4314-85dc-1b735eaff9e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:04.025158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80896bad-c89c-4314-85dc-1b735eaff9e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:04.026591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80896bad-c89c-4314-85dc-1b735eaff9e1 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:04.036364 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 982/1987] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:39:04.044854 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1006/1988] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:03 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:04.049588 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80896bad-c89c-4314-85dc-1b735eaff9e1 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:04.060924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80896bad-c89c-4314-85dc-1b735eaff9e1 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:04.135775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbe145-0aca-4cd8-80bd-afdcb98e4949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:04.136470 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbe145-0aca-4cd8-80bd-afdcb98e4949 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:04.136764 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbe145-0aca-4cd8-80bd-afdcb98e4949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:04.137051 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbe145-0aca-4cd8-80bd-afdcb98e4949 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:04.137758 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cbe145-0aca-4cd8-80bd-afdcb98e4949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:04.139739 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cbe145-0aca-4cd8-80bd-afdcb98e4949 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:04.199744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1c375-8a0a-45e4-b62f-7f9b09a1b2d3 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:04.258399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:04.262598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 983/1989] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:04 2022] POST /identity/v3/users => generated 290 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:04.269228 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbe145-0aca-4cd8-80bd-afdcb98e4949 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:04.276816 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1c375-8a0a-45e4-b62f-7f9b09a1b2d3 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:04.284687 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbe145-0aca-4cd8-80bd-afdcb98e4949 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:04.288009 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb7b8a-c933-4eaf-8362-e72c05667711 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:04.289712 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb7b8a-c933-4eaf-8362-e72c05667711 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:04.290190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efb434-37b5-4cd3-ad26-1e5e50c22979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:04.294018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb7b8a-c933-4eaf-8362-e72c05667711 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:04.294018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb7b8a-c933-4eaf-8362-e72c05667711 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:04.294227 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efb434-37b5-4cd3-ad26-1e5e50c22979 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:04.294227 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efb434-37b5-4cd3-ad26-1e5e50c22979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:04.294227 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efb434-37b5-4cd3-ad26-1e5e50c22979 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:04.294227 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efb434-37b5-4cd3-ad26-1e5e50c22979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:04.294637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3cdb7b8a-c933-4eaf-8362-e72c05667711 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:04.294732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efb434-37b5-4cd3-ad26-1e5e50c22979 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:04.306608 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 984/1990] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:04.313080 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efb434-37b5-4cd3-ad26-1e5e50c22979 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:04.325573 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efb434-37b5-4cd3-ad26-1e5e50c22979 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:04.331654 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b3dcd-9d06-46aa-9886-71dce741970f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:04.332404 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b3dcd-9d06-46aa-9886-71dce741970f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:04.332829 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b3dcd-9d06-46aa-9886-71dce741970f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:04.333156 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b3dcd-9d06-46aa-9886-71dce741970f admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:04.333907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b3dcd-9d06-46aa-9886-71dce741970f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:04.335656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b3dcd-9d06-46aa-9886-71dce741970f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:04.345941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:04.351308 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 985/1991] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:04.355966 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b3dcd-9d06-46aa-9886-71dce741970f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:04.367266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b3dcd-9d06-46aa-9886-71dce741970f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:04.374422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab68d7-a411-4398-ace7-6e8d7e74492f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:04.375564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab68d7-a411-4398-ace7-6e8d7e74492f admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:04.375854 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab68d7-a411-4398-ace7-6e8d7e74492f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:04.376138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab68d7-a411-4398-ace7-6e8d7e74492f admin admin] PATH_INFO: `/v3/projects/d2f419c46cbd454599b0fc7e6c112e11/users/69a0ea60d0fe4d5ab81c12f0920b861e/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:04.384306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb7b8a-c933-4eaf-8362-e72c05667711 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:04.385812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb7b8a-c933-4eaf-8362-e72c05667711 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:04.394394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab68d7-a411-4398-ace7-6e8d7e74492f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2f419c46cbd454599b0fc7e6c112e11, user_id=69a0ea60d0fe4d5ab81c12f0920b861e, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:04.396364 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1007/1992] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:39:04.397872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab68d7-a411-4398-ace7-6e8d7e74492f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:04.517856 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 986/1993] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:04 2022] PUT /identity/v3/projects/d2f419c46cbd454599b0fc7e6c112e11/users/69a0ea60d0fe4d5ab81c12f0920b861e/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:04.523263 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcbea3-bf41-4d5d-b15c-d8468fadc0e2 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:04.523547 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcbea3-bf41-4d5d-b15c-d8468fadc0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:04.523884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcbea3-bf41-4d5d-b15c-d8468fadc0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:04.554858 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7bfcbea3-bf41-4d5d-b15c-d8468fadc0e2 None None] MFA Rules not processed for user `69a0ea60d0fe4d5ab81c12f0920b861e`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:04.556268 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7bfcbea3-bf41-4d5d-b15c-d8468fadc0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:04.636703 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1008/1994] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:04.648037 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab68d7-a411-4398-ace7-6e8d7e74492f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:04.723309 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab68d7-a411-4398-ace7-6e8d7e74492f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:04.729017 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd917e12-3f87-4393-aacc-2d84cd5d2ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:04.729695 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd917e12-3f87-4393-aacc-2d84cd5d2ddc admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:04.729981 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd917e12-3f87-4393-aacc-2d84cd5d2ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:04.730459 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd917e12-3f87-4393-aacc-2d84cd5d2ddc admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:04.731293 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd917e12-3f87-4393-aacc-2d84cd5d2ddc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:04.733374 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd917e12-3f87-4393-aacc-2d84cd5d2ddc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:04.737556 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:04.846495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 987/1995] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:04 2022] POST /identity/v3/projects => generated 379 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:04.850875 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfcbea3-bf41-4d5d-b15c-d8468fadc0e2 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:04.863659 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfcbea3-bf41-4d5d-b15c-d8468fadc0e2 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:04.894871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:04.941417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:04.942381 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05b0c1-be54-45a1-ba3b-676865b8a67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:04.943804 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05b0c1-be54-45a1-ba3b-676865b8a67e admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:04.944274 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05b0c1-be54-45a1-ba3b-676865b8a67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:04.944752 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05b0c1-be54-45a1-ba3b-676865b8a67e admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:04.946159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05b0c1-be54-45a1-ba3b-676865b8a67e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:04.949764 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05b0c1-be54-45a1-ba3b-676865b8a67e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:05.047047 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1009/1996] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:04 2022] POST /identity/v3/users => generated 296 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:05.051773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd917e12-3f87-4393-aacc-2d84cd5d2ddc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:05.064908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd917e12-3f87-4393-aacc-2d84cd5d2ddc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:05.070013 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03941f51-d5d9-4d6b-beb4-01c2e0be182e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:05.070939 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03941f51-d5d9-4d6b-beb4-01c2e0be182e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:05.071163 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03941f51-d5d9-4d6b-beb4-01c2e0be182e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:05.071445 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03941f51-d5d9-4d6b-beb4-01c2e0be182e admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:05.072180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03941f51-d5d9-4d6b-beb4-01c2e0be182e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:05.073856 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03941f51-d5d9-4d6b-beb4-01c2e0be182e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:05.090816 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 988/1997] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:05.092414 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05b0c1-be54-45a1-ba3b-676865b8a67e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:05.105797 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05b0c1-be54-45a1-ba3b-676865b8a67e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:05.112698 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639bc324-63d7-45ac-bccc-328518d615f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:05.114277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639bc324-63d7-45ac-bccc-328518d615f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:05.114764 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639bc324-63d7-45ac-bccc-328518d615f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:05.115233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639bc324-63d7-45ac-bccc-328518d615f1 admin admin] PATH_INFO: `/v3/projects/c20a7cf580af4c289f7984c9de7a38b6/users/f5a4b105b7f14c21b58b825fe41fe188/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:05.134060 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639bc324-63d7-45ac-bccc-328518d615f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c20a7cf580af4c289f7984c9de7a38b6, user_id=f5a4b105b7f14c21b58b825fe41fe188, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:05.136742 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639bc324-63d7-45ac-bccc-328518d615f1 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:05.174808 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:05.237779 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1010/1998] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:05 2022] PUT /identity/v3/projects/c20a7cf580af4c289f7984c9de7a38b6/users/f5a4b105b7f14c21b58b825fe41fe188/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:39:05.241871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03941f51-d5d9-4d6b-beb4-01c2e0be182e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:05.252065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03941f51-d5d9-4d6b-beb4-01c2e0be182e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:05.326474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89253a2e-7865-42b1-aea5-fccbc58a78ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:05.327073 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89253a2e-7865-42b1-aea5-fccbc58a78ee admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:05.327360 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89253a2e-7865-42b1-aea5-fccbc58a78ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:05.327640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89253a2e-7865-42b1-aea5-fccbc58a78ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:05.328500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89253a2e-7865-42b1-aea5-fccbc58a78ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:05.330186 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89253a2e-7865-42b1-aea5-fccbc58a78ee admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:05.341950 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 989/1999] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:05.348330 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639bc324-63d7-45ac-bccc-328518d615f1 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:05.364297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639bc324-63d7-45ac-bccc-328518d615f1 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:05.371785 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7ccba-8280-4177-8fcc-06a9620fb930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:05.372769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7ccba-8280-4177-8fcc-06a9620fb930 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:05.373038 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7ccba-8280-4177-8fcc-06a9620fb930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:05.373313 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7ccba-8280-4177-8fcc-06a9620fb930 admin admin] PATH_INFO: `/v3/projects/c20a7cf580af4c289f7984c9de7a38b6/users/f5a4b105b7f14c21b58b825fe41fe188/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:05.378414 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec7ccba-8280-4177-8fcc-06a9620fb930 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c20a7cf580af4c289f7984c9de7a38b6, user_id=f5a4b105b7f14c21b58b825fe41fe188, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:05.384023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec7ccba-8280-4177-8fcc-06a9620fb930 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:05.402301 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1011/2000] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:05 2022] PUT /identity/v3/projects/c20a7cf580af4c289f7984c9de7a38b6/users/f5a4b105b7f14c21b58b825fe41fe188/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:05.413782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263fd79-7711-4463-ae7d-2903df0f0379 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:05.415023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263fd79-7711-4463-ae7d-2903df0f0379 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:05.415023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263fd79-7711-4463-ae7d-2903df0f0379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:05.455257 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7263fd79-7711-4463-ae7d-2903df0f0379 None None] MFA Rules not processed for user `f5a4b105b7f14c21b58b825fe41fe188`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:05.456596 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7263fd79-7711-4463-ae7d-2903df0f0379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:05.513516 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:05.555143 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 990/2001] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:05.637939 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7ccba-8280-4177-8fcc-06a9620fb930 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:05.639784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2ec7ccba-8280-4177-8fcc-06a9620fb930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:05.716007 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7ccba-8280-4177-8fcc-06a9620fb930 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:05.721393 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc9b67-22dc-438b-ae5e-39a645945c89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:05.722047 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc9b67-22dc-438b-ae5e-39a645945c89 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:05.722332 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc9b67-22dc-438b-ae5e-39a645945c89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:05.722607 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc9b67-22dc-438b-ae5e-39a645945c89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:05.734727 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc9b67-22dc-438b-ae5e-39a645945c89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:05.736500 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc9b67-22dc-438b-ae5e-39a645945c89 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:05.742275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:05.748990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1012/2002] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:06.272458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:06.607801 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263fd79-7711-4463-ae7d-2903df0f0379 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:06.617706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:DELETE=8,keystone:SELECT=87,keystone:INSERT=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:06.718823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:06.734528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263fd79-7711-4463-ae7d-2903df0f0379 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:06.741537 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3727e-75bd-4842-a3c9-1273d33e2582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:06.742528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3727e-75bd-4842-a3c9-1273d33e2582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:06.742884 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3727e-75bd-4842-a3c9-1273d33e2582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:06.743266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3727e-75bd-4842-a3c9-1273d33e2582 admin admin] PATH_INFO: `/v3/users/69a0ea60d0fe4d5ab81c12f0920b861e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:06.744863 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3727e-75bd-4842-a3c9-1273d33e2582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69a0ea60d0fe4d5ab81c12f0920b861e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:06.746931 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3727e-75bd-4842-a3c9-1273d33e2582 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:06.937842 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eee3727e-75bd-4842-a3c9-1273d33e2582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69a0ea60d0fe4d5ab81c12f0920b861e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:06.940138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eee3727e-75bd-4842-a3c9-1273d33e2582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69a0ea60d0fe4d5ab81c12f0920b861e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:06.948931 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eee3727e-75bd-4842-a3c9-1273d33e2582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69a0ea60d0fe4d5ab81c12f0920b861e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:06.951219 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eee3727e-75bd-4842-a3c9-1273d33e2582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69a0ea60d0fe4d5ab81c12f0920b861e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:06.963211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eee3727e-75bd-4842-a3c9-1273d33e2582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69a0ea60d0fe4d5ab81c12f0920b861e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:07.082738 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 991/2003] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:06 2022] DELETE /identity/v3/users/69a0ea60d0fe4d5ab81c12f0920b861e => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:07.087953 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7208014-e1d8-43ca-b6ee-007467be1b2f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:07.088248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7208014-e1d8-43ca-b6ee-007467be1b2f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:07.088534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7208014-e1d8-43ca-b6ee-007467be1b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:07.112392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e7208014-e1d8-43ca-b6ee-007467be1b2f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:07.112392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e7208014-e1d8-43ca-b6ee-007467be1b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:07.177161 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3727e-75bd-4842-a3c9-1273d33e2582 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:07.180475 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-eee3727e-75bd-4842-a3c9-1273d33e2582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:07.189191 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1013/2004] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:07.199791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7208014-e1d8-43ca-b6ee-007467be1b2f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:07.205613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e7208014-e1d8-43ca-b6ee-007467be1b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:07.205723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3727e-75bd-4842-a3c9-1273d33e2582 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:07.207301 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2051e76-3b9a-41e1-ae2f-0e5d8525b5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:07.208277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2051e76-3b9a-41e1-ae2f-0e5d8525b5fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:07.208629 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2051e76-3b9a-41e1-ae2f-0e5d8525b5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:07.209246 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2051e76-3b9a-41e1-ae2f-0e5d8525b5fb admin admin] PATH_INFO: `/v3/users/ad4bdd987a814c639dd020c6244ab06b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:07.210381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2051e76-3b9a-41e1-ae2f-0e5d8525b5fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad4bdd987a814c639dd020c6244ab06b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:07.212532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2051e76-3b9a-41e1-ae2f-0e5d8525b5fb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:07.275558 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7208014-e1d8-43ca-b6ee-007467be1b2f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:07.281079 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66849ee7-ce93-4b6a-93db-03c4e9aa0177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:07.281754 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66849ee7-ce93-4b6a-93db-03c4e9aa0177 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:07.282052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66849ee7-ce93-4b6a-93db-03c4e9aa0177 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:07.282320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66849ee7-ce93-4b6a-93db-03c4e9aa0177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:07.293478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66849ee7-ce93-4b6a-93db-03c4e9aa0177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:07.295104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66849ee7-ce93-4b6a-93db-03c4e9aa0177 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:07.305562 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1014/2005] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:39:07.352783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:07.515263 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e2051e76-3b9a-41e1-ae2f-0e5d8525b5fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad4bdd987a814c639dd020c6244ab06b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:07.516280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e2051e76-3b9a-41e1-ae2f-0e5d8525b5fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad4bdd987a814c639dd020c6244ab06b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:07.532210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e2051e76-3b9a-41e1-ae2f-0e5d8525b5fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad4bdd987a814c639dd020c6244ab06b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:07.533539 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e2051e76-3b9a-41e1-ae2f-0e5d8525b5fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad4bdd987a814c639dd020c6244ab06b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:07.542761 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:07.553122 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e2051e76-3b9a-41e1-ae2f-0e5d8525b5fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad4bdd987a814c639dd020c6244ab06b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:07.617938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 992/2006] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:07 2022] DELETE /identity/v3/users/ad4bdd987a814c639dd020c6244ab06b => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:07.624664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba6c05-95e1-415e-80d5-582b6b35412a None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:07.625206 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba6c05-95e1-415e-80d5-582b6b35412a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:07.625697 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba6c05-95e1-415e-80d5-582b6b35412a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:07.649824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-83ba6c05-95e1-415e-80d5-582b6b35412a None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:07.651302 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-83ba6c05-95e1-415e-80d5-582b6b35412a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:07.789765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1015/2007] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:07.802746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2051e76-3b9a-41e1-ae2f-0e5d8525b5fb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:07.806466 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e2051e76-3b9a-41e1-ae2f-0e5d8525b5fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:07.869849 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba6c05-95e1-415e-80d5-582b6b35412a None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:07.872657 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-83ba6c05-95e1-415e-80d5-582b6b35412a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:07.899551 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba6c05-95e1-415e-80d5-582b6b35412a None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:07.907678 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf07ea4-37ac-4d3c-b671-8458515c59c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:07.910125 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf07ea4-37ac-4d3c-b671-8458515c59c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:07.910412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf07ea4-37ac-4d3c-b671-8458515c59c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:07.910686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf07ea4-37ac-4d3c-b671-8458515c59c4 admin admin] PATH_INFO: `/v3/projects/d2f419c46cbd454599b0fc7e6c112e11` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:07.912227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf07ea4-37ac-4d3c-b671-8458515c59c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2f419c46cbd454599b0fc7e6c112e11)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:07.918011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf07ea4-37ac-4d3c-b671-8458515c59c4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:07.924313 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2051e76-3b9a-41e1-ae2f-0e5d8525b5fb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:07.941318 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5326955-c0d9-4246-bc02-94a753b4cfc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:07.947188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5326955-c0d9-4246-bc02-94a753b4cfc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:07.947188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5326955-c0d9-4246-bc02-94a753b4cfc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:07.947188 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5326955-c0d9-4246-bc02-94a753b4cfc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:07.960504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5326955-c0d9-4246-bc02-94a753b4cfc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:07.962252 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5326955-c0d9-4246-bc02-94a753b4cfc1 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:07.976463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 993/2008] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:08.012284 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:08.038291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2bf07ea4-37ac-4d3c-b671-8458515c59c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2f419c46cbd454599b0fc7e6c112e11', 'request_id': 'req-2bf07ea4-37ac-4d3c-b671-8458515c59c4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:08.050758 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2bf07ea4-37ac-4d3c-b671-8458515c59c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2f419c46cbd454599b0fc7e6c112e11', 'request_id': 'req-2bf07ea4-37ac-4d3c-b671-8458515c59c4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:08.088831 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:08.203576 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:08.205122 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2bf07ea4-37ac-4d3c-b671-8458515c59c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2f419c46cbd454599b0fc7e6c112e11', 'request_id': 'req-2bf07ea4-37ac-4d3c-b671-8458515c59c4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:08.216860 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2bf07ea4-37ac-4d3c-b671-8458515c59c4 admin admin] The token cache is being invalidate because project d2f419c46cbd454599b0fc7e6c112e11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:08.217178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2bf07ea4-37ac-4d3c-b671-8458515c59c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:08.217837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2bf07ea4-37ac-4d3c-b671-8458515c59c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:08.219599 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1016/2009] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:07 2022] DELETE /identity/v3/projects/d2f419c46cbd454599b0fc7e6c112e11 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:08.223416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5326955-c0d9-4246-bc02-94a753b4cfc1 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:08.234084 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5326955-c0d9-4246-bc02-94a753b4cfc1 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:08.298296 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a61de6-f217-4b0b-a47f-da73ddfe0b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:08.301545 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a61de6-f217-4b0b-a47f-da73ddfe0b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:08.302206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a61de6-f217-4b0b-a47f-da73ddfe0b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:08.303570 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a61de6-f217-4b0b-a47f-da73ddfe0b70 admin admin] PATH_INFO: `/v3/users/f5a4b105b7f14c21b58b825fe41fe188` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:08.306404 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a61de6-f217-4b0b-a47f-da73ddfe0b70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5a4b105b7f14c21b58b825fe41fe188)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:08.311754 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a61de6-f217-4b0b-a47f-da73ddfe0b70 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:08.530642 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f1a61de6-f217-4b0b-a47f-da73ddfe0b70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5a4b105b7f14c21b58b825fe41fe188'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:08.531673 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f1a61de6-f217-4b0b-a47f-da73ddfe0b70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5a4b105b7f14c21b58b825fe41fe188'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:08.539789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f1a61de6-f217-4b0b-a47f-da73ddfe0b70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5a4b105b7f14c21b58b825fe41fe188'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:08.545938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f1a61de6-f217-4b0b-a47f-da73ddfe0b70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5a4b105b7f14c21b58b825fe41fe188'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:08.554555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f1a61de6-f217-4b0b-a47f-da73ddfe0b70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5a4b105b7f14c21b58b825fe41fe188'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:08.575654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:08.586895 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf07ea4-37ac-4d3c-b671-8458515c59c4 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:08.588978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2bf07ea4-37ac-4d3c-b671-8458515c59c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:08.604326 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf07ea4-37ac-4d3c-b671-8458515c59c4 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:08.616250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb69b0-a8ab-4d87-9115-6ea4cabc6c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:08.616250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb69b0-a8ab-4d87-9115-6ea4cabc6c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:08.616250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb69b0-a8ab-4d87-9115-6ea4cabc6c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:08.616250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb69b0-a8ab-4d87-9115-6ea4cabc6c4f admin admin] PATH_INFO: `/v3/projects/62ac248ca98d4b59828a0acc450e299d` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:08.617058 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb69b0-a8ab-4d87-9115-6ea4cabc6c4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=62ac248ca98d4b59828a0acc450e299d)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:08.617353 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb69b0-a8ab-4d87-9115-6ea4cabc6c4f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:08.685178 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:08.690689 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 994/2010] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:08 2022] DELETE /identity/v3/users/f5a4b105b7f14c21b58b825fe41fe188 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:08.766003 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-dafb69b0-a8ab-4d87-9115-6ea4cabc6c4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62ac248ca98d4b59828a0acc450e299d', 'request_id': 'req-dafb69b0-a8ab-4d87-9115-6ea4cabc6c4f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:08.775502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-dafb69b0-a8ab-4d87-9115-6ea4cabc6c4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62ac248ca98d4b59828a0acc450e299d', 'request_id': 'req-dafb69b0-a8ab-4d87-9115-6ea4cabc6c4f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:08.812219 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a61de6-f217-4b0b-a47f-da73ddfe0b70 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:08.814752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f1a61de6-f217-4b0b-a47f-da73ddfe0b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:08.875892 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-dafb69b0-a8ab-4d87-9115-6ea4cabc6c4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62ac248ca98d4b59828a0acc450e299d', 'request_id': 'req-dafb69b0-a8ab-4d87-9115-6ea4cabc6c4f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:08.892635 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a61de6-f217-4b0b-a47f-da73ddfe0b70 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:08.898689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-dafb69b0-a8ab-4d87-9115-6ea4cabc6c4f admin admin] The token cache is being invalidate because project 62ac248ca98d4b59828a0acc450e299d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:08.899038 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-dafb69b0-a8ab-4d87-9115-6ea4cabc6c4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:08.899891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-dafb69b0-a8ab-4d87-9115-6ea4cabc6c4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:08.902132 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1017/2011] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:08 2022] DELETE /identity/v3/projects/62ac248ca98d4b59828a0acc450e299d => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:39:08.907525 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb69b0-a8ab-4d87-9115-6ea4cabc6c4f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:08.909363 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-dafb69b0-a8ab-4d87-9115-6ea4cabc6c4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:08.917270 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:08.984655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42d0df-cb18-4241-a387-76dbe4f1db50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:08.986441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42d0df-cb18-4241-a387-76dbe4f1db50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:08.986792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42d0df-cb18-4241-a387-76dbe4f1db50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:08.987263 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42d0df-cb18-4241-a387-76dbe4f1db50 admin admin] PATH_INFO: `/v3/users/7f2f0abc6fac4084a9eaa364497a0b51` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:08.989818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42d0df-cb18-4241-a387-76dbe4f1db50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f2f0abc6fac4084a9eaa364497a0b51)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:08.994691 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42d0df-cb18-4241-a387-76dbe4f1db50 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:09.030156 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:09.060387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb69b0-a8ab-4d87-9115-6ea4cabc6c4f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:09.069969 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbf2f4-d463-4f6e-b682-d3aaf45e3012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:09.071034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbf2f4-d463-4f6e-b682-d3aaf45e3012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:09.071313 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbf2f4-d463-4f6e-b682-d3aaf45e3012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:09.071591 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbf2f4-d463-4f6e-b682-d3aaf45e3012 admin admin] PATH_INFO: `/v3/users/83d08d82c20046d693391c8b1a0701cb` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:09.072903 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbf2f4-d463-4f6e-b682-d3aaf45e3012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83d08d82c20046d693391c8b1a0701cb)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:09.075509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbf2f4-d463-4f6e-b682-d3aaf45e3012 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:09.178094 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8b42d0df-cb18-4241-a387-76dbe4f1db50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f2f0abc6fac4084a9eaa364497a0b51'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:09.178900 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8b42d0df-cb18-4241-a387-76dbe4f1db50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f2f0abc6fac4084a9eaa364497a0b51'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:09.186764 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8b42d0df-cb18-4241-a387-76dbe4f1db50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f2f0abc6fac4084a9eaa364497a0b51'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:09.188102 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8b42d0df-cb18-4241-a387-76dbe4f1db50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f2f0abc6fac4084a9eaa364497a0b51'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:09.203712 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8b42d0df-cb18-4241-a387-76dbe4f1db50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f2f0abc6fac4084a9eaa364497a0b51'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:09.274045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 995/2012] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:08 2022] DELETE /identity/v3/users/7f2f0abc6fac4084a9eaa364497a0b51 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:39:09.279265 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cf45f-a7bf-44f5-ab4e-278a40c43b53 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:09.279613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cf45f-a7bf-44f5-ab4e-278a40c43b53 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:09.279896 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cf45f-a7bf-44f5-ab4e-278a40c43b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:09.288304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-46cbf2f4-d463-4f6e-b682-d3aaf45e3012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83d08d82c20046d693391c8b1a0701cb'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:09.289453 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-46cbf2f4-d463-4f6e-b682-d3aaf45e3012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83d08d82c20046d693391c8b1a0701cb'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:09.297951 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-46cbf2f4-d463-4f6e-b682-d3aaf45e3012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83d08d82c20046d693391c8b1a0701cb'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:09.298875 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-46cbf2f4-d463-4f6e-b682-d3aaf45e3012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83d08d82c20046d693391c8b1a0701cb'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:09.306253 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-425cf45f-a7bf-44f5-ab4e-278a40c43b53 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:09.307508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-425cf45f-a7bf-44f5-ab4e-278a40c43b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:09.376354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 996/2013] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:09.376354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cf45f-a7bf-44f5-ab4e-278a40c43b53 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:09.380511 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-425cf45f-a7bf-44f5-ab4e-278a40c43b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:09.385936 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-46cbf2f4-d463-4f6e-b682-d3aaf45e3012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83d08d82c20046d693391c8b1a0701cb'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:09.393394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cf45f-a7bf-44f5-ab4e-278a40c43b53 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:09.406376 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1018/2014] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:08 2022] DELETE /identity/v3/users/83d08d82c20046d693391c8b1a0701cb => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:39:09.407534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbf2f4-d463-4f6e-b682-d3aaf45e3012 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:09.408600 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f89868-97be-4c7d-b0ad-494753dcfe92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:09.409410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-46cbf2f4-d463-4f6e-b682-d3aaf45e3012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:09.420125 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f89868-97be-4c7d-b0ad-494753dcfe92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:09.421264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f89868-97be-4c7d-b0ad-494753dcfe92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:09.421581 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f89868-97be-4c7d-b0ad-494753dcfe92 admin admin] PATH_INFO: `/v3/projects/c20a7cf580af4c289f7984c9de7a38b6` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:09.423005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f89868-97be-4c7d-b0ad-494753dcfe92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c20a7cf580af4c289f7984c9de7a38b6)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:09.425309 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f89868-97be-4c7d-b0ad-494753dcfe92 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:09.503153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbf2f4-d463-4f6e-b682-d3aaf45e3012 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:09.507831 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631bb83-044b-43b5-8658-2b070ed3250d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:09.508649 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631bb83-044b-43b5-8658-2b070ed3250d service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:09.508938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631bb83-044b-43b5-8658-2b070ed3250d service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:09.509247 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631bb83-044b-43b5-8658-2b070ed3250d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:09.561587 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a7f89868-97be-4c7d-b0ad-494753dcfe92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c20a7cf580af4c289f7984c9de7a38b6', 'request_id': 'req-a7f89868-97be-4c7d-b0ad-494753dcfe92'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:09.567221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a7f89868-97be-4c7d-b0ad-494753dcfe92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c20a7cf580af4c289f7984c9de7a38b6', 'request_id': 'req-a7f89868-97be-4c7d-b0ad-494753dcfe92'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:09.574671 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a7f89868-97be-4c7d-b0ad-494753dcfe92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c20a7cf580af4c289f7984c9de7a38b6', 'request_id': 'req-a7f89868-97be-4c7d-b0ad-494753dcfe92'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:09.581402 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631bb83-044b-43b5-8658-2b070ed3250d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:09.582858 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631bb83-044b-43b5-8658-2b070ed3250d service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:09.593598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1019/2015] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:09.594514 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631bb83-044b-43b5-8658-2b070ed3250d service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:09.595918 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7631bb83-044b-43b5-8658-2b070ed3250d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:09.610601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631bb83-044b-43b5-8658-2b070ed3250d service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:09.680957 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a7f89868-97be-4c7d-b0ad-494753dcfe92 admin admin] The token cache is being invalidate because project c20a7cf580af4c289f7984c9de7a38b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:09.681447 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a7f89868-97be-4c7d-b0ad-494753dcfe92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:09.682479 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a7f89868-97be-4c7d-b0ad-494753dcfe92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:09.685046 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 997/2016] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:09 2022] DELETE /identity/v3/projects/c20a7cf580af4c289f7984c9de7a38b6 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:09.691305 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15b153-6765-49f9-b6f4-ca6f5d434749 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:09.692245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15b153-6765-49f9-b6f4-ca6f5d434749 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:09.693154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15b153-6765-49f9-b6f4-ca6f5d434749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:09.709334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1fcbb-8e29-4c23-929c-49ae8f63a56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:09.709334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1fcbb-8e29-4c23-929c-49ae8f63a56d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:09.709334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1fcbb-8e29-4c23-929c-49ae8f63a56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:09.709334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1fcbb-8e29-4c23-929c-49ae8f63a56d admin admin] PATH_INFO: `/v3/users/787c95aa412847eaa09ecf735829dcaa` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:09.711429 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1fcbb-8e29-4c23-929c-49ae8f63a56d admin admin] RBAC: Authorizing `identity:delete_user(user_id=787c95aa412847eaa09ecf735829dcaa)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:09.715686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1fcbb-8e29-4c23-929c-49ae8f63a56d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:09.734343 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1c15b153-6765-49f9-b6f4-ca6f5d434749 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:09.736269 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1c15b153-6765-49f9-b6f4-ca6f5d434749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:09.749162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 998/2017] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:09.757588 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c51ae-5baa-49e8-81eb-2ae7bf2268cd None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:09.757696 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c51ae-5baa-49e8-81eb-2ae7bf2268cd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:09.757904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c51ae-5baa-49e8-81eb-2ae7bf2268cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:09.768728 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:09.777097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ae8c51ae-5baa-49e8-81eb-2ae7bf2268cd None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:09.778396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ae8c51ae-5baa-49e8-81eb-2ae7bf2268cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:09.787197 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 999/2018] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:09.791493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c51ae-5baa-49e8-81eb-2ae7bf2268cd None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:09.801299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c51ae-5baa-49e8-81eb-2ae7bf2268cd None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:09.806793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b71d2c-7733-46fe-926d-e96db7c084eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:09.807478 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b71d2c-7733-46fe-926d-e96db7c084eb admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:09.807766 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b71d2c-7733-46fe-926d-e96db7c084eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:09.808044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b71d2c-7733-46fe-926d-e96db7c084eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:09.808783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b71d2c-7733-46fe-926d-e96db7c084eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:39:09.809131 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b71d2c-7733-46fe-926d-e96db7c084eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:09.810785 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b71d2c-7733-46fe-926d-e96db7c084eb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:09.824566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1000/2019] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:39:09.829085 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b71d2c-7733-46fe-926d-e96db7c084eb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:09.839241 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b71d2c-7733-46fe-926d-e96db7c084eb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:09.844637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da41ff-f35b-41c7-b43f-b6a676621bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:09.845401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da41ff-f35b-41c7-b43f-b6a676621bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:09.850374 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da41ff-f35b-41c7-b43f-b6a676621bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:09.850374 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da41ff-f35b-41c7-b43f-b6a676621bb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:09.850374 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da41ff-f35b-41c7-b43f-b6a676621bb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:09.850374 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da41ff-f35b-41c7-b43f-b6a676621bb2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:09.870612 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:09.934706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2eb1fcbb-8e29-4c23-929c-49ae8f63a56d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '787c95aa412847eaa09ecf735829dcaa'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:09.935406 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2eb1fcbb-8e29-4c23-929c-49ae8f63a56d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '787c95aa412847eaa09ecf735829dcaa'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:09.944565 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2eb1fcbb-8e29-4c23-929c-49ae8f63a56d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '787c95aa412847eaa09ecf735829dcaa'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:09.945906 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2eb1fcbb-8e29-4c23-929c-49ae8f63a56d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '787c95aa412847eaa09ecf735829dcaa'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:10.258498 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1001/2020] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:09 2022] POST /identity/v3/projects => generated 375 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:10.265552 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da41ff-f35b-41c7-b43f-b6a676621bb2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:10.270065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f9da41ff-f35b-41c7-b43f-b6a676621bb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:10.328540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2eb1fcbb-8e29-4c23-929c-49ae8f63a56d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '787c95aa412847eaa09ecf735829dcaa'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:10.368083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da41ff-f35b-41c7-b43f-b6a676621bb2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:10.372151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1020/2021] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:09 2022] DELETE /identity/v3/users/787c95aa412847eaa09ecf735829dcaa => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:10.373810 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1fcbb-8e29-4c23-929c-49ae8f63a56d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:10.375266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2eb1fcbb-8e29-4c23-929c-49ae8f63a56d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:10.375394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef029b31-d409-4cc5-8b26-a454e52460f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:10.376431 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef029b31-d409-4cc5-8b26-a454e52460f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:10.376692 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef029b31-d409-4cc5-8b26-a454e52460f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:10.377131 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef029b31-d409-4cc5-8b26-a454e52460f5 admin admin] PATH_INFO: `/v3/projects/f078798a21d7423ebcd371fe37adb8ca` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:10.389128 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef029b31-d409-4cc5-8b26-a454e52460f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f078798a21d7423ebcd371fe37adb8ca)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:10.394401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef029b31-d409-4cc5-8b26-a454e52460f5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:10.469291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1fcbb-8e29-4c23-929c-49ae8f63a56d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:10.477798 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04969012-822b-48db-b989-e37815dd8f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:10.479349 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04969012-822b-48db-b989-e37815dd8f49 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:10.479872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04969012-822b-48db-b989-e37815dd8f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:10.480656 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04969012-822b-48db-b989-e37815dd8f49 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:10.481409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04969012-822b-48db-b989-e37815dd8f49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:10.483392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04969012-822b-48db-b989-e37815dd8f49 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:10.557806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ef029b31-d409-4cc5-8b26-a454e52460f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f078798a21d7423ebcd371fe37adb8ca', 'request_id': 'req-ef029b31-d409-4cc5-8b26-a454e52460f5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:10.566770 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ef029b31-d409-4cc5-8b26-a454e52460f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f078798a21d7423ebcd371fe37adb8ca', 'request_id': 'req-ef029b31-d409-4cc5-8b26-a454e52460f5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:10.578695 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ef029b31-d409-4cc5-8b26-a454e52460f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f078798a21d7423ebcd371fe37adb8ca', 'request_id': 'req-ef029b31-d409-4cc5-8b26-a454e52460f5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:10.606789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1021/2022] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:10 2022] POST /identity/v3/users => generated 288 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:10.608329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0467857-6f85-41b2-b3c1-0a7f5f3659cd None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:10.608627 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0467857-6f85-41b2-b3c1-0a7f5f3659cd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:10.608909 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0467857-6f85-41b2-b3c1-0a7f5f3659cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:10.642988 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b0467857-6f85-41b2-b3c1-0a7f5f3659cd None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:10.644501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b0467857-6f85-41b2-b3c1-0a7f5f3659cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:10.648356 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ef029b31-d409-4cc5-8b26-a454e52460f5 admin admin] The token cache is being invalidate because project f078798a21d7423ebcd371fe37adb8ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:10.652652 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ef029b31-d409-4cc5-8b26-a454e52460f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:10.652652 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ef029b31-d409-4cc5-8b26-a454e52460f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:10.652652 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1002/2023] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:10 2022] DELETE /identity/v3/projects/f078798a21d7423ebcd371fe37adb8ca => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:10.652652 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef029b31-d409-4cc5-8b26-a454e52460f5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:10.653175 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ef029b31-d409-4cc5-8b26-a454e52460f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:10.720396 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1022/2024] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:10.720396 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0467857-6f85-41b2-b3c1-0a7f5f3659cd None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:10.720396 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b0467857-6f85-41b2-b3c1-0a7f5f3659cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:10.724383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef029b31-d409-4cc5-8b26-a454e52460f5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:10.730762 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f74157-701f-4167-a81d-20aaf18c3a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:10.731655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f74157-701f-4167-a81d-20aaf18c3a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:10.731946 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f74157-701f-4167-a81d-20aaf18c3a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:10.732137 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0467857-6f85-41b2-b3c1-0a7f5f3659cd None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:10.732248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f74157-701f-4167-a81d-20aaf18c3a74 admin admin] PATH_INFO: `/v3/projects/e2baf2dbac8b438aaec4e9ceb078c387` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:10.733716 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f74157-701f-4167-a81d-20aaf18c3a74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2baf2dbac8b438aaec4e9ceb078c387)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:10.735836 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f74157-701f-4167-a81d-20aaf18c3a74 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:10.737569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9d1ab-dfdb-48de-b512-15485edcd5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:10.739814 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9d1ab-dfdb-48de-b512-15485edcd5fb admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:10.739814 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9d1ab-dfdb-48de-b512-15485edcd5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:10.739980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9d1ab-dfdb-48de-b512-15485edcd5fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:10.741362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c9d1ab-dfdb-48de-b512-15485edcd5fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:10.743185 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c9d1ab-dfdb-48de-b512-15485edcd5fb admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:10.758098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1023/2025] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:10.759929 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc116cf-5178-42f8-9ed0-35bbd866b920 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:10.760222 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc116cf-5178-42f8-9ed0-35bbd866b920 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:10.760505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc116cf-5178-42f8-9ed0-35bbd866b920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:10.780670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-acc116cf-5178-42f8-9ed0-35bbd866b920 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:10.781921 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-acc116cf-5178-42f8-9ed0-35bbd866b920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:10.790916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1024/2026] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:10.791772 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc116cf-5178-42f8-9ed0-35bbd866b920 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:10.793160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-acc116cf-5178-42f8-9ed0-35bbd866b920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:10.859580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc116cf-5178-42f8-9ed0-35bbd866b920 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:10.861554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f3f74157-701f-4167-a81d-20aaf18c3a74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2baf2dbac8b438aaec4e9ceb078c387', 'request_id': 'req-f3f74157-701f-4167-a81d-20aaf18c3a74'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:10.874350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f3f74157-701f-4167-a81d-20aaf18c3a74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2baf2dbac8b438aaec4e9ceb078c387', 'request_id': 'req-f3f74157-701f-4167-a81d-20aaf18c3a74'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:10.880821 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f3f74157-701f-4167-a81d-20aaf18c3a74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2baf2dbac8b438aaec4e9ceb078c387', 'request_id': 'req-f3f74157-701f-4167-a81d-20aaf18c3a74'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:10.931972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc9e38-97dc-4ba8-813a-8fd0da320ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:10.932687 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc9e38-97dc-4ba8-813a-8fd0da320ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:10.932977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc9e38-97dc-4ba8-813a-8fd0da320ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:10.933308 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc9e38-97dc-4ba8-813a-8fd0da320ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:10.993805 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f3f74157-701f-4167-a81d-20aaf18c3a74 admin admin] The token cache is being invalidate because project e2baf2dbac8b438aaec4e9ceb078c387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:10.994256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f3f74157-701f-4167-a81d-20aaf18c3a74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:10.995084 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f3f74157-701f-4167-a81d-20aaf18c3a74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:10.996706 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1003/2027] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:10 2022] DELETE /identity/v3/projects/e2baf2dbac8b438aaec4e9ceb078c387 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:10.997548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f74157-701f-4167-a81d-20aaf18c3a74 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:10.999471 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f3f74157-701f-4167-a81d-20aaf18c3a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:11.002358 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc9e38-97dc-4ba8-813a-8fd0da320ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:11.003782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc9e38-97dc-4ba8-813a-8fd0da320ec1 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:11.013232 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f74157-701f-4167-a81d-20aaf18c3a74 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:11.014487 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1025/2028] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:11.015965 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd5ab0-478f-45cc-bd25-b1e7adedcbe5 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:11.016360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd5ab0-478f-45cc-bd25-b1e7adedcbe5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:11.016641 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd5ab0-478f-45cc-bd25-b1e7adedcbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:11.027077 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63ec6f-6926-40b3-aff5-31021fbbd5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:11.027827 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63ec6f-6926-40b3-aff5-31021fbbd5cf admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:11.028174 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63ec6f-6926-40b3-aff5-31021fbbd5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:11.028516 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63ec6f-6926-40b3-aff5-31021fbbd5cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:11.029684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b63ec6f-6926-40b3-aff5-31021fbbd5cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:11.031891 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b63ec6f-6926-40b3-aff5-31021fbbd5cf admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:11.042983 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-fabd5ab0-478f-45cc-bd25-b1e7adedcbe5 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:11.044265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-fabd5ab0-478f-45cc-bd25-b1e7adedcbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:11.045922 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1004/2029] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:11.046546 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63ec6f-6926-40b3-aff5-31021fbbd5cf admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:11.048420 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7b63ec6f-6926-40b3-aff5-31021fbbd5cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:11.056819 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1026/2030] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:11.060642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd5ab0-478f-45cc-bd25-b1e7adedcbe5 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:11.063871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63ec6f-6926-40b3-aff5-31021fbbd5cf admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:11.069751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b716b4-4661-42d3-822d-ab4496fab0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:11.070715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b716b4-4661-42d3-822d-ab4496fab0d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:11.070998 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b716b4-4661-42d3-822d-ab4496fab0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:11.071390 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd5ab0-478f-45cc-bd25-b1e7adedcbe5 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:11.071778 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b716b4-4661-42d3-822d-ab4496fab0d6 admin admin] PATH_INFO: `/v3/users/9b090a32f18a46569a6930d264422a58` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:11.073176 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b716b4-4661-42d3-822d-ab4496fab0d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b090a32f18a46569a6930d264422a58)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:11.079047 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b716b4-4661-42d3-822d-ab4496fab0d6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:11.079182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c622c6-4ffe-4b32-877f-dab1688bc99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:11.079182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c622c6-4ffe-4b32-877f-dab1688bc99f admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:11.079182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c622c6-4ffe-4b32-877f-dab1688bc99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:11.079182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c622c6-4ffe-4b32-877f-dab1688bc99f admin admin] PATH_INFO: `/v3/projects/a2b6cf5dc2584b8aa03904869a1ef92e/users/2c5023c3a1b0471e961b6ca563c380da/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:11.097226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c622c6-4ffe-4b32-877f-dab1688bc99f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b6cf5dc2584b8aa03904869a1ef92e, user_id=2c5023c3a1b0471e961b6ca563c380da, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:11.101950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c622c6-4ffe-4b32-877f-dab1688bc99f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:11.251392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1027/2031] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:11 2022] PUT /identity/v3/projects/a2b6cf5dc2584b8aa03904869a1ef92e/users/2c5023c3a1b0471e961b6ca563c380da/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:11.251850 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c622c6-4ffe-4b32-877f-dab1688bc99f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:11.285942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:11.332650 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c622c6-4ffe-4b32-877f-dab1688bc99f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:11.340772 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73bd70-04ea-44d9-8dcc-7dfcde877f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:11.341485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73bd70-04ea-44d9-8dcc-7dfcde877f4d admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:11.341833 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73bd70-04ea-44d9-8dcc-7dfcde877f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:11.342111 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73bd70-04ea-44d9-8dcc-7dfcde877f4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:11.342858 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73bd70-04ea-44d9-8dcc-7dfcde877f4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:39:11.343167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73bd70-04ea-44d9-8dcc-7dfcde877f4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:11.345953 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73bd70-04ea-44d9-8dcc-7dfcde877f4d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:11.347630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:11.348979 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f0b716b4-4661-42d3-822d-ab4496fab0d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b090a32f18a46569a6930d264422a58'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:11.352274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f0b716b4-4661-42d3-822d-ab4496fab0d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b090a32f18a46569a6930d264422a58'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:11.365258 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1028/2032] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:39:11.365258 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1578c-c143-4f5c-965b-7ff66426c618 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:11.365258 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1578c-c143-4f5c-965b-7ff66426c618 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:11.365258 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1578c-c143-4f5c-965b-7ff66426c618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:11.367350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f0b716b4-4661-42d3-822d-ab4496fab0d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b090a32f18a46569a6930d264422a58'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:11.368195 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f0b716b4-4661-42d3-822d-ab4496fab0d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b090a32f18a46569a6930d264422a58'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:11.384729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f0b716b4-4661-42d3-822d-ab4496fab0d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b090a32f18a46569a6930d264422a58'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:11.397427 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e0f1578c-c143-4f5c-965b-7ff66426c618 None None] MFA Rules not processed for user `2c5023c3a1b0471e961b6ca563c380da`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:11.399627 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e0f1578c-c143-4f5c-965b-7ff66426c618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:11.472000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1005/2033] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:11 2022] DELETE /identity/v3/users/9b090a32f18a46569a6930d264422a58 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:11.472817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b716b4-4661-42d3-822d-ab4496fab0d6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:11.474271 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f0b716b4-4661-42d3-822d-ab4496fab0d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:11.485807 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1029/2034] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:11.490223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1578c-c143-4f5c-965b-7ff66426c618 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:11.491629 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e0f1578c-c143-4f5c-965b-7ff66426c618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:11.507653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1578c-c143-4f5c-965b-7ff66426c618 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:11.573664 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b716b4-4661-42d3-822d-ab4496fab0d6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:11.579916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb5164-a623-4b9f-911a-3e1154670333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:11.580610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb5164-a623-4b9f-911a-3e1154670333 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:11.580887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb5164-a623-4b9f-911a-3e1154670333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:11.581085 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed7ad5-1ace-4a81-a14f-c2db498a9f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:11.581264 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb5164-a623-4b9f-911a-3e1154670333 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:11.581847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed7ad5-1ace-4a81-a14f-c2db498a9f42 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:11.582011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb5164-a623-4b9f-911a-3e1154670333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:11.582346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed7ad5-1ace-4a81-a14f-c2db498a9f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:11.583183 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed7ad5-1ace-4a81-a14f-c2db498a9f42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:11.584454 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed7ad5-1ace-4a81-a14f-c2db498a9f42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:11.586522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed7ad5-1ace-4a81-a14f-c2db498a9f42 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:11.591516 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb5164-a623-4b9f-911a-3e1154670333 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:11.748367 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1006/2035] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:11 2022] POST /identity/v3/projects => generated 367 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:11.751136 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed7ad5-1ace-4a81-a14f-c2db498a9f42 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:11.754192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-17ed7ad5-1ace-4a81-a14f-c2db498a9f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:11.779996 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1030/2036] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:11 2022] POST /identity/v3/projects => generated 373 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:11.788429 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb5164-a623-4b9f-911a-3e1154670333 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:11.845957 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed7ad5-1ace-4a81-a14f-c2db498a9f42 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:11.862620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb5164-a623-4b9f-911a-3e1154670333 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:11.868926 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fa59f-68ba-433b-b40a-982ec843149c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:11.869632 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fa59f-68ba-433b-b40a-982ec843149c admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:11.870377 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fa59f-68ba-433b-b40a-982ec843149c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:11.870377 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fa59f-68ba-433b-b40a-982ec843149c admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:11.871018 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2fa59f-68ba-433b-b40a-982ec843149c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:11.872957 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2fa59f-68ba-433b-b40a-982ec843149c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:11.888731 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac019aec-a34b-4b7e-ab58-fc46dcada3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:11.889654 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac019aec-a34b-4b7e-ab58-fc46dcada3b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:11.889938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac019aec-a34b-4b7e-ab58-fc46dcada3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:11.890217 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac019aec-a34b-4b7e-ab58-fc46dcada3b9 admin admin] PATH_INFO: `/v3/projects/c8d8f5253c9241b0a3ca4e79db14f7ed` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:11.891412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac019aec-a34b-4b7e-ab58-fc46dcada3b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8d8f5253c9241b0a3ca4e79db14f7ed)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:11.893426 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac019aec-a34b-4b7e-ab58-fc46dcada3b9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:12.002858 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1031/2037] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:11 2022] POST /identity/v3/users => generated 284 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:12.002858 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fa59f-68ba-433b-b40a-982ec843149c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:12.011023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fa59f-68ba-433b-b40a-982ec843149c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:12.015760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be580c5-0190-45ea-9fec-7b9bf5c11cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:12.016427 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be580c5-0190-45ea-9fec-7b9bf5c11cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:12.016703 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be580c5-0190-45ea-9fec-7b9bf5c11cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:12.016973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be580c5-0190-45ea-9fec-7b9bf5c11cd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:12.017685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be580c5-0190-45ea-9fec-7b9bf5c11cd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:12.041804 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be580c5-0190-45ea-9fec-7b9bf5c11cd5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:12.055848 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ac019aec-a34b-4b7e-ab58-fc46dcada3b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8d8f5253c9241b0a3ca4e79db14f7ed', 'request_id': 'req-ac019aec-a34b-4b7e-ab58-fc46dcada3b9'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:12.062267 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ac019aec-a34b-4b7e-ab58-fc46dcada3b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8d8f5253c9241b0a3ca4e79db14f7ed', 'request_id': 'req-ac019aec-a34b-4b7e-ab58-fc46dcada3b9'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:12.069601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ac019aec-a34b-4b7e-ab58-fc46dcada3b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8d8f5253c9241b0a3ca4e79db14f7ed', 'request_id': 'req-ac019aec-a34b-4b7e-ab58-fc46dcada3b9'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:12.160521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1032/2038] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:11 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:12.162943 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be580c5-0190-45ea-9fec-7b9bf5c11cd5 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:12.175094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be580c5-0190-45ea-9fec-7b9bf5c11cd5 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:12.209607 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ac019aec-a34b-4b7e-ab58-fc46dcada3b9 admin admin] The token cache is being invalidate because project c8d8f5253c9241b0a3ca4e79db14f7ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:12.210075 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ac019aec-a34b-4b7e-ab58-fc46dcada3b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:12.210883 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ac019aec-a34b-4b7e-ab58-fc46dcada3b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:12.212696 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1007/2039] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:11 2022] DELETE /identity/v3/projects/c8d8f5253c9241b0a3ca4e79db14f7ed => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:12.213594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac019aec-a34b-4b7e-ab58-fc46dcada3b9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:12.214948 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ac019aec-a34b-4b7e-ab58-fc46dcada3b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:12.244273 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446e8ea-691a-4a09-b101-2e5aa2aafccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:12.244948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446e8ea-691a-4a09-b101-2e5aa2aafccd admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:12.247116 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446e8ea-691a-4a09-b101-2e5aa2aafccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:12.247116 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446e8ea-691a-4a09-b101-2e5aa2aafccd admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:12.247116 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446e8ea-691a-4a09-b101-2e5aa2aafccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:12.248045 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446e8ea-691a-4a09-b101-2e5aa2aafccd admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:12.258476 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1033/2040] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:12.264623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446e8ea-691a-4a09-b101-2e5aa2aafccd admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:12.264623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8446e8ea-691a-4a09-b101-2e5aa2aafccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:12.276747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446e8ea-691a-4a09-b101-2e5aa2aafccd admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:12.287774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57823113-6e6b-4322-99c8-f58a8289bd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:12.287774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57823113-6e6b-4322-99c8-f58a8289bd3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:12.287774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57823113-6e6b-4322-99c8-f58a8289bd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:12.287774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57823113-6e6b-4322-99c8-f58a8289bd3c admin admin] PATH_INFO: `/v3/users/8666735682d648d1958a7db76bb98672` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:12.288135 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57823113-6e6b-4322-99c8-f58a8289bd3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8666735682d648d1958a7db76bb98672)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:12.288135 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57823113-6e6b-4322-99c8-f58a8289bd3c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:12.295052 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac019aec-a34b-4b7e-ab58-fc46dcada3b9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:12.300955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d53ea1a-3525-48e6-8b23-af72e785238b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:12.301704 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53ea1a-3525-48e6-8b23-af72e785238b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:12.301991 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53ea1a-3525-48e6-8b23-af72e785238b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:12.302271 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53ea1a-3525-48e6-8b23-af72e785238b admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:12.303561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d53ea1a-3525-48e6-8b23-af72e785238b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:12.305390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d53ea1a-3525-48e6-8b23-af72e785238b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:12.315759 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1008/2041] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:12.320200 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d53ea1a-3525-48e6-8b23-af72e785238b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:12.320200 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0d53ea1a-3525-48e6-8b23-af72e785238b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:12.331931 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d53ea1a-3525-48e6-8b23-af72e785238b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:12.337355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff10c8-46e8-4863-86a0-0968feda7d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:12.338080 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff10c8-46e8-4863-86a0-0968feda7d5e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:12.338365 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff10c8-46e8-4863-86a0-0968feda7d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:12.338644 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff10c8-46e8-4863-86a0-0968feda7d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:12.339399 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff10c8-46e8-4863-86a0-0968feda7d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:12.341058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff10c8-46e8-4863-86a0-0968feda7d5e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:12.352240 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1009/2042] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:12.353103 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff10c8-46e8-4863-86a0-0968feda7d5e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:12.354802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-96ff10c8-46e8-4863-86a0-0968feda7d5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:12.368674 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff10c8-46e8-4863-86a0-0968feda7d5e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:12.373450 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a1688c-be8a-46c7-af03-3c33eeea7617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:12.383372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a1688c-be8a-46c7-af03-3c33eeea7617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:12.383372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a1688c-be8a-46c7-af03-3c33eeea7617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:12.383372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a1688c-be8a-46c7-af03-3c33eeea7617 admin admin] PATH_INFO: `/v3/projects/195d6601aef54c468ed9d24d291ec82a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:12.383372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a1688c-be8a-46c7-af03-3c33eeea7617 admin admin] RBAC: Authorizing `identity:delete_project(project_id=195d6601aef54c468ed9d24d291ec82a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:12.383372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a1688c-be8a-46c7-af03-3c33eeea7617 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:12.460073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:12.575309 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:12.595614 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-57823113-6e6b-4322-99c8-f58a8289bd3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8666735682d648d1958a7db76bb98672'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:12.595614 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-57823113-6e6b-4322-99c8-f58a8289bd3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8666735682d648d1958a7db76bb98672'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:12.595913 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-16a1688c-be8a-46c7-af03-3c33eeea7617 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '195d6601aef54c468ed9d24d291ec82a', 'request_id': 'req-16a1688c-be8a-46c7-af03-3c33eeea7617'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:12.597970 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-57823113-6e6b-4322-99c8-f58a8289bd3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8666735682d648d1958a7db76bb98672'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:12.598659 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-57823113-6e6b-4322-99c8-f58a8289bd3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8666735682d648d1958a7db76bb98672'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:12.610156 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-16a1688c-be8a-46c7-af03-3c33eeea7617 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '195d6601aef54c468ed9d24d291ec82a', 'request_id': 'req-16a1688c-be8a-46c7-af03-3c33eeea7617'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:12.626834 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-16a1688c-be8a-46c7-af03-3c33eeea7617 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '195d6601aef54c468ed9d24d291ec82a', 'request_id': 'req-16a1688c-be8a-46c7-af03-3c33eeea7617'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:12.710861 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-57823113-6e6b-4322-99c8-f58a8289bd3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8666735682d648d1958a7db76bb98672'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:12.728250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1034/2043] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:12 2022] DELETE /identity/v3/users/8666735682d648d1958a7db76bb98672 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:12.729166 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57823113-6e6b-4322-99c8-f58a8289bd3c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:12.730498 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-57823113-6e6b-4322-99c8-f58a8289bd3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:12.785511 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-16a1688c-be8a-46c7-af03-3c33eeea7617 admin admin] The token cache is being invalidate because project 195d6601aef54c468ed9d24d291ec82a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:12.785780 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-16a1688c-be8a-46c7-af03-3c33eeea7617 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:12.786553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-16a1688c-be8a-46c7-af03-3c33eeea7617 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:12.788343 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1010/2044] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:12 2022] DELETE /identity/v3/projects/195d6601aef54c468ed9d24d291ec82a => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:12.789557 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a1688c-be8a-46c7-af03-3c33eeea7617 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:12.791104 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-16a1688c-be8a-46c7-af03-3c33eeea7617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:12.798485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57823113-6e6b-4322-99c8-f58a8289bd3c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:12.804348 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2203a-a501-4ce3-aa0c-c95fdab5cce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:12.805547 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2203a-a501-4ce3-aa0c-c95fdab5cce0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:12.805824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2203a-a501-4ce3-aa0c-c95fdab5cce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:12.806102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2203a-a501-4ce3-aa0c-c95fdab5cce0 admin admin] PATH_INFO: `/v3/projects/993055fe255948b38a39bd8e79bb1ad8/users/4e61b774e49c47d5adf60d422c9d0970/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:12.824276 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2203a-a501-4ce3-aa0c-c95fdab5cce0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=993055fe255948b38a39bd8e79bb1ad8, user_id=4e61b774e49c47d5adf60d422c9d0970, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:12.827048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2203a-a501-4ce3-aa0c-c95fdab5cce0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:12.874295 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a1688c-be8a-46c7-af03-3c33eeea7617 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:12.879863 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ccee13-c154-4941-bc1e-4bffc78e952f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:12.881191 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccee13-c154-4941-bc1e-4bffc78e952f admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:12.881663 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccee13-c154-4941-bc1e-4bffc78e952f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:12.881944 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccee13-c154-4941-bc1e-4bffc78e952f admin admin] PATH_INFO: `/v3/projects/d2432f00db7f413b88df47d89276cb05/users/11f54ae8d51c49489891bf4fa6841d63/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:12.902892 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1035/2045] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:12 2022] PUT /identity/v3/projects/993055fe255948b38a39bd8e79bb1ad8/users/4e61b774e49c47d5adf60d422c9d0970/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:12.907992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ccee13-c154-4941-bc1e-4bffc78e952f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2432f00db7f413b88df47d89276cb05, user_id=11f54ae8d51c49489891bf4fa6841d63, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:12.909683 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ccee13-c154-4941-bc1e-4bffc78e952f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:12.914628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa495158-917b-4850-8278-894b4da967ab None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:12.915213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa495158-917b-4850-8278-894b4da967ab None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:12.915213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa495158-917b-4850-8278-894b4da967ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:12.935669 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-fa495158-917b-4850-8278-894b4da967ab None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:12.940590 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-fa495158-917b-4850-8278-894b4da967ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:13.018624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1011/2046] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:12 2022] PUT /identity/v3/projects/d2432f00db7f413b88df47d89276cb05/users/11f54ae8d51c49489891bf4fa6841d63/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:13.025901 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ccee13-c154-4941-bc1e-4bffc78e952f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:13.027368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-77ccee13-c154-4941-bc1e-4bffc78e952f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:13.059306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1036/2047] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:13.067790 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee4392-04da-4dc1-b7fb-b0036a01001a None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:13.068207 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee4392-04da-4dc1-b7fb-b0036a01001a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:13.068639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee4392-04da-4dc1-b7fb-b0036a01001a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:13.122881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-79ee4392-04da-4dc1-b7fb-b0036a01001a None None] MFA Rules not processed for user `11f54ae8d51c49489891bf4fa6841d63`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:13.130189 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-79ee4392-04da-4dc1-b7fb-b0036a01001a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:13.170839 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ccee13-c154-4941-bc1e-4bffc78e952f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:13.178963 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:13.187381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec9544-8ba9-4ea4-9946-6e81a64cfbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:13.188120 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec9544-8ba9-4ea4-9946-6e81a64cfbda admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:13.188408 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec9544-8ba9-4ea4-9946-6e81a64cfbda admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:13.188690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec9544-8ba9-4ea4-9946-6e81a64cfbda admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:13.189572 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec9544-8ba9-4ea4-9946-6e81a64cfbda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:13.193396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec9544-8ba9-4ea4-9946-6e81a64cfbda admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:13.214300 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1012/2048] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:13.216042 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e75a96-36f5-4369-9fe0-9d2254adaa38 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:13.216406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e75a96-36f5-4369-9fe0-9d2254adaa38 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:13.216824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e75a96-36f5-4369-9fe0-9d2254adaa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:13.217626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1037/2049] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:13 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:13.223810 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee4392-04da-4dc1-b7fb-b0036a01001a None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:13.235744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee4392-04da-4dc1-b7fb-b0036a01001a None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:13.238839 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-47e75a96-36f5-4369-9fe0-9d2254adaa38 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:13.245329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-47e75a96-36f5-4369-9fe0-9d2254adaa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:13.251215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1013/2050] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:13.252202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e75a96-36f5-4369-9fe0-9d2254adaa38 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:13.254455 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6aa8b-2552-49a2-a72f-b34783814ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:13.255736 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6aa8b-2552-49a2-a72f-b34783814ecd admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:13.256240 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6aa8b-2552-49a2-a72f-b34783814ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:13.256715 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6aa8b-2552-49a2-a72f-b34783814ecd admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:13.257908 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc6aa8b-2552-49a2-a72f-b34783814ecd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:13.261218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc6aa8b-2552-49a2-a72f-b34783814ecd admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:13.263505 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e75a96-36f5-4369-9fe0-9d2254adaa38 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:13.269100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30bb78-9cdd-48ce-bf49-e7d04b75fd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:13.270003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30bb78-9cdd-48ce-bf49-e7d04b75fd18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:13.274913 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30bb78-9cdd-48ce-bf49-e7d04b75fd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:13.274913 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30bb78-9cdd-48ce-bf49-e7d04b75fd18 admin admin] PATH_INFO: `/v3/projects/993055fe255948b38a39bd8e79bb1ad8/users/4e61b774e49c47d5adf60d422c9d0970/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:13.278520 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30bb78-9cdd-48ce-bf49-e7d04b75fd18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=993055fe255948b38a39bd8e79bb1ad8, user_id=4e61b774e49c47d5adf60d422c9d0970, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:13.281239 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30bb78-9cdd-48ce-bf49-e7d04b75fd18 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:13.301240 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1014/2051] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:13 2022] PUT /identity/v3/projects/993055fe255948b38a39bd8e79bb1ad8/users/4e61b774e49c47d5adf60d422c9d0970/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:13.301240 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30bb78-9cdd-48ce-bf49-e7d04b75fd18 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:13.370457 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:13.381357 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1038/2052] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:13 2022] POST /identity/v3/projects => generated 367 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:13.381357 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6aa8b-2552-49a2-a72f-b34783814ecd admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:13.381357 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9fc6aa8b-2552-49a2-a72f-b34783814ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:13.385691 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30bb78-9cdd-48ce-bf49-e7d04b75fd18 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:13.454549 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099edb7-8e69-49ad-8e69-9f4cc53f4668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:13.455434 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099edb7-8e69-49ad-8e69-9f4cc53f4668 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:13.455776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099edb7-8e69-49ad-8e69-9f4cc53f4668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:13.456064 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099edb7-8e69-49ad-8e69-9f4cc53f4668 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:13.457326 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099edb7-8e69-49ad-8e69-9f4cc53f4668 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:39:13.457653 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099edb7-8e69-49ad-8e69-9f4cc53f4668 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:13.459949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099edb7-8e69-49ad-8e69-9f4cc53f4668 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:13.473074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1015/2053] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:39:13.474489 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f6980-6068-4b15-958d-d9e22ab9b335 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:13.475353 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f6980-6068-4b15-958d-d9e22ab9b335 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:13.475640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f6980-6068-4b15-958d-d9e22ab9b335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:13.479022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6aa8b-2552-49a2-a72f-b34783814ecd admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:13.488645 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe0a44-78e2-4881-978f-e4bd88cffdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:13.490945 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe0a44-78e2-4881-978f-e4bd88cffdf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:13.490945 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe0a44-78e2-4881-978f-e4bd88cffdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:13.490945 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe0a44-78e2-4881-978f-e4bd88cffdf8 admin admin] PATH_INFO: `/v3/projects/365dc71d274044d1985699f6501c7099` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:13.492140 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe0a44-78e2-4881-978f-e4bd88cffdf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=365dc71d274044d1985699f6501c7099)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:13.494882 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe0a44-78e2-4881-978f-e4bd88cffdf8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:13.504806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5c3f6980-6068-4b15-958d-d9e22ab9b335 None None] MFA Rules not processed for user `4e61b774e49c47d5adf60d422c9d0970`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:13.506162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5c3f6980-6068-4b15-958d-d9e22ab9b335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:13.583124 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1016/2054] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:13.584500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f6980-6068-4b15-958d-d9e22ab9b335 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:13.595575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f6980-6068-4b15-958d-d9e22ab9b335 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:13.600075 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834f332-b907-4372-b125-cda41945caa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:13.600867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834f332-b907-4372-b125-cda41945caa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:13.601183 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834f332-b907-4372-b125-cda41945caa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:13.601457 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834f332-b907-4372-b125-cda41945caa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:13.602138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834f332-b907-4372-b125-cda41945caa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:13.604220 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834f332-b907-4372-b125-cda41945caa5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:13.658705 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1ebe0a44-78e2-4881-978f-e4bd88cffdf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '365dc71d274044d1985699f6501c7099', 'request_id': 'req-1ebe0a44-78e2-4881-978f-e4bd88cffdf8'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:13.665406 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1ebe0a44-78e2-4881-978f-e4bd88cffdf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '365dc71d274044d1985699f6501c7099', 'request_id': 'req-1ebe0a44-78e2-4881-978f-e4bd88cffdf8'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:13.718026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1017/2055] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:13 2022] POST /identity/v3/users => generated 290 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:13.719013 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834f332-b907-4372-b125-cda41945caa5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:13.730665 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834f332-b907-4372-b125-cda41945caa5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:13.741213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1ebe0a44-78e2-4881-978f-e4bd88cffdf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '365dc71d274044d1985699f6501c7099', 'request_id': 'req-1ebe0a44-78e2-4881-978f-e4bd88cffdf8'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:13.753511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1ebe0a44-78e2-4881-978f-e4bd88cffdf8 admin admin] The token cache is being invalidate because project 365dc71d274044d1985699f6501c7099 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:13.753806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1ebe0a44-78e2-4881-978f-e4bd88cffdf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:13.754651 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1ebe0a44-78e2-4881-978f-e4bd88cffdf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:13.757274 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1039/2056] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:13 2022] DELETE /identity/v3/projects/365dc71d274044d1985699f6501c7099 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:13.758266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe0a44-78e2-4881-978f-e4bd88cffdf8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:13.760062 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1ebe0a44-78e2-4881-978f-e4bd88cffdf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:13.799518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1fb74-0e3d-4844-be11-8d0178959640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:13.800177 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1fb74-0e3d-4844-be11-8d0178959640 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:13.800460 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1fb74-0e3d-4844-be11-8d0178959640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:13.800791 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1fb74-0e3d-4844-be11-8d0178959640 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:13.801527 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1fb74-0e3d-4844-be11-8d0178959640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:13.803662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1fb74-0e3d-4844-be11-8d0178959640 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:13.831705 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe0a44-78e2-4881-978f-e4bd88cffdf8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:13.836072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428cc52-77fd-40c3-a047-d5697d3f40bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:13.836861 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428cc52-77fd-40c3-a047-d5697d3f40bd service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:13.837167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428cc52-77fd-40c3-a047-d5697d3f40bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:13.837441 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428cc52-77fd-40c3-a047-d5697d3f40bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:13.839280 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4428cc52-77fd-40c3-a047-d5697d3f40bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:13.881189 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1018/2057] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:13 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:13.882308 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1fb74-0e3d-4844-be11-8d0178959640 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:13.885970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1ba1fb74-0e3d-4844-be11-8d0178959640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:13.909080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4428cc52-77fd-40c3-a047-d5697d3f40bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:13.910516 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4428cc52-77fd-40c3-a047-d5697d3f40bd service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:13.911598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:13.930147 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1040/2058] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:13.930995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428cc52-77fd-40c3-a047-d5697d3f40bd service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:13.932522 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4428cc52-77fd-40c3-a047-d5697d3f40bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:13.950761 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428cc52-77fd-40c3-a047-d5697d3f40bd service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:13.951587 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda4f76-3ef2-4020-99b8-2dd741e76063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:13.952759 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda4f76-3ef2-4020-99b8-2dd741e76063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:13.953078 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda4f76-3ef2-4020-99b8-2dd741e76063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:13.953357 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda4f76-3ef2-4020-99b8-2dd741e76063 admin admin] PATH_INFO: `/v3/users/cb5a3abc3797419e82eb045cf0487d52` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:13.954696 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda4f76-3ef2-4020-99b8-2dd741e76063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb5a3abc3797419e82eb045cf0487d52)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:13.955870 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1fb74-0e3d-4844-be11-8d0178959640 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:13.956628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda4f76-3ef2-4020-99b8-2dd741e76063 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:13.961223 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634695b-42ef-44c0-b84b-b3118fe77679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:13.962031 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634695b-42ef-44c0-b84b-b3118fe77679 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:13.962168 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634695b-42ef-44c0-b84b-b3118fe77679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:13.962614 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634695b-42ef-44c0-b84b-b3118fe77679 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:13.963350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634695b-42ef-44c0-b84b-b3118fe77679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:13.965217 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634695b-42ef-44c0-b84b-b3118fe77679 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:13.981443 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1019/2059] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:13.981443 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634695b-42ef-44c0-b84b-b3118fe77679 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:13.984597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e634695b-42ef-44c0-b84b-b3118fe77679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:14.004897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634695b-42ef-44c0-b84b-b3118fe77679 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:14.013818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca830a9f-62ab-4527-9410-1c15ed933c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:14.014918 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca830a9f-62ab-4527-9410-1c15ed933c30 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:14.015248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca830a9f-62ab-4527-9410-1c15ed933c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:14.015529 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca830a9f-62ab-4527-9410-1c15ed933c30 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:14.016797 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca830a9f-62ab-4527-9410-1c15ed933c30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:14.020278 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca830a9f-62ab-4527-9410-1c15ed933c30 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:14.047231 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:14.109039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1020/2060] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:13 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:14.109039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca830a9f-62ab-4527-9410-1c15ed933c30 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:14.128324 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca830a9f-62ab-4527-9410-1c15ed933c30 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:14.134366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd744b-9c11-4cda-abf7-5a3d971c59ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:14.135448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd744b-9c11-4cda-abf7-5a3d971c59ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:14.135864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd744b-9c11-4cda-abf7-5a3d971c59ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:14.136268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd744b-9c11-4cda-abf7-5a3d971c59ef admin admin] PATH_INFO: `/v3/projects/f3f7f33ad81045038af1adf000797229/users/8f1ec8b7b13c4662b8581944115090b8/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:14.145101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4cda4f76-3ef2-4020-99b8-2dd741e76063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb5a3abc3797419e82eb045cf0487d52'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:14.152991 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4cda4f76-3ef2-4020-99b8-2dd741e76063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb5a3abc3797419e82eb045cf0487d52'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:14.153174 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd744b-9c11-4cda-abf7-5a3d971c59ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3f7f33ad81045038af1adf000797229, user_id=8f1ec8b7b13c4662b8581944115090b8, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:14.155742 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd744b-9c11-4cda-abf7-5a3d971c59ef admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:14.166640 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4cda4f76-3ef2-4020-99b8-2dd741e76063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb5a3abc3797419e82eb045cf0487d52'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:14.167635 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4cda4f76-3ef2-4020-99b8-2dd741e76063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb5a3abc3797419e82eb045cf0487d52'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:14.283391 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4cda4f76-3ef2-4020-99b8-2dd741e76063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb5a3abc3797419e82eb045cf0487d52'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:14.298764 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1021/2061] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:14 2022] PUT /identity/v3/projects/f3f7f33ad81045038af1adf000797229/users/8f1ec8b7b13c4662b8581944115090b8/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:14.299854 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd744b-9c11-4cda-abf7-5a3d971c59ef admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:14.302464 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7bfd744b-9c11-4cda-abf7-5a3d971c59ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:14.316080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1041/2062] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:13 2022] DELETE /identity/v3/users/cb5a3abc3797419e82eb045cf0487d52 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:14.321120 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda4f76-3ef2-4020-99b8-2dd741e76063 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:14.322181 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4cda4f76-3ef2-4020-99b8-2dd741e76063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:14.408330 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd744b-9c11-4cda-abf7-5a3d971c59ef admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:14.413881 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cb2d7-9ab0-4589-905f-480ad3b2a19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:14.414705 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cb2d7-9ab0-4589-905f-480ad3b2a19d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:14.415086 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cb2d7-9ab0-4589-905f-480ad3b2a19d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:14.415799 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cb2d7-9ab0-4589-905f-480ad3b2a19d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:14.416910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cb2d7-9ab0-4589-905f-480ad3b2a19d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:14.418718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cb2d7-9ab0-4589-905f-480ad3b2a19d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:14.426095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda4f76-3ef2-4020-99b8-2dd741e76063 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:14.433519 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1022/2063] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:14.434771 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cb2d7-9ab0-4589-905f-480ad3b2a19d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:14.437439 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a123d-e7a1-4b56-b944-6ec73219f0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:14.439124 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a123d-e7a1-4b56-b944-6ec73219f0de admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:14.440009 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a123d-e7a1-4b56-b944-6ec73219f0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:14.441317 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a123d-e7a1-4b56-b944-6ec73219f0de admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:14.443436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a123d-e7a1-4b56-b944-6ec73219f0de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:14.445094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a123d-e7a1-4b56-b944-6ec73219f0de admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:14.448681 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cb2d7-9ab0-4589-905f-480ad3b2a19d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:14.454302 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014bc04-6664-4892-b9c8-537a652ab23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:14.459228 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014bc04-6664-4892-b9c8-537a652ab23a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:14.459228 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014bc04-6664-4892-b9c8-537a652ab23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:14.459228 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014bc04-6664-4892-b9c8-537a652ab23a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:14.459228 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014bc04-6664-4892-b9c8-537a652ab23a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:14.459811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1042/2064] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:14.461594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014bc04-6664-4892-b9c8-537a652ab23a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:14.462541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a123d-e7a1-4b56-b944-6ec73219f0de admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:14.477781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1023/2065] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:14.478823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014bc04-6664-4892-b9c8-537a652ab23a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:14.486392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a123d-e7a1-4b56-b944-6ec73219f0de admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:14.489555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014bc04-6664-4892-b9c8-537a652ab23a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:14.492520 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d122ff4-c377-4052-a958-3cfd43564b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:14.493719 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d122ff4-c377-4052-a958-3cfd43564b29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:14.494103 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d122ff4-c377-4052-a958-3cfd43564b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:14.494577 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d122ff4-c377-4052-a958-3cfd43564b29 admin admin] PATH_INFO: `/v3/projects/f3f7f33ad81045038af1adf000797229/users/8f1ec8b7b13c4662b8581944115090b8/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:14.495873 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b242a-c1dc-4713-af3a-ca48ac7ce959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:14.496871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b242a-c1dc-4713-af3a-ca48ac7ce959 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:14.497221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b242a-c1dc-4713-af3a-ca48ac7ce959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:14.497411 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d122ff4-c377-4052-a958-3cfd43564b29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3f7f33ad81045038af1adf000797229, user_id=8f1ec8b7b13c4662b8581944115090b8, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:14.497526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b242a-c1dc-4713-af3a-ca48ac7ce959 admin admin] PATH_INFO: `/v3/projects/242eef5928b14c1691d35ba2f13d7fbf/users/2ae9de8e4a1646b3a6bd5c51a63ecb9b/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:14.500440 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d122ff4-c377-4052-a958-3cfd43564b29 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:14.513228 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1043/2066] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:14 2022] PUT /identity/v3/projects/f3f7f33ad81045038af1adf000797229/users/8f1ec8b7b13c4662b8581944115090b8/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:14.529210 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156358c7-6535-4b8d-ab52-40fc193050e5 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:14.529587 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156358c7-6535-4b8d-ab52-40fc193050e5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:14.529704 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b242a-c1dc-4713-af3a-ca48ac7ce959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=242eef5928b14c1691d35ba2f13d7fbf, user_id=2ae9de8e4a1646b3a6bd5c51a63ecb9b, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:14.530095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156358c7-6535-4b8d-ab52-40fc193050e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:14.532315 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b242a-c1dc-4713-af3a-ca48ac7ce959 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:14.571983 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-156358c7-6535-4b8d-ab52-40fc193050e5 None None] MFA Rules not processed for user `8f1ec8b7b13c4662b8581944115090b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:14.573297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-156358c7-6535-4b8d-ab52-40fc193050e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:14.633645 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1024/2067] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:14 2022] PUT /identity/v3/projects/242eef5928b14c1691d35ba2f13d7fbf/users/2ae9de8e4a1646b3a6bd5c51a63ecb9b/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:14.649049 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22af2d-7507-4774-9ca7-3ba8f810b53e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:14.649437 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22af2d-7507-4774-9ca7-3ba8f810b53e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:14.650103 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22af2d-7507-4774-9ca7-3ba8f810b53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:14.678253 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1044/2068] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:14 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:14.725869 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156358c7-6535-4b8d-ab52-40fc193050e5 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:14.725869 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-156358c7-6535-4b8d-ab52-40fc193050e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:14.750619 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fc22af2d-7507-4774-9ca7-3ba8f810b53e None None] MFA Rules not processed for user `2ae9de8e4a1646b3a6bd5c51a63ecb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:14.752364 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fc22af2d-7507-4774-9ca7-3ba8f810b53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:14.845229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156358c7-6535-4b8d-ab52-40fc193050e5 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:14.882903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1025/2069] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:14.897934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22af2d-7507-4774-9ca7-3ba8f810b53e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:14.903921 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:14.903921 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fc22af2d-7507-4774-9ca7-3ba8f810b53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:14.937480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90629e1-1200-4fc7-b3ea-898f723402ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:14.938563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90629e1-1200-4fc7-b3ea-898f723402ce service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:14.938941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90629e1-1200-4fc7-b3ea-898f723402ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:14.939434 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90629e1-1200-4fc7-b3ea-898f723402ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:15.024931 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22af2d-7507-4774-9ca7-3ba8f810b53e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:15.031161 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc95dd-9010-4aa8-ba59-7b63d28aba1d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4d9b5e3826049f2800f1b9424977d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:15.031836 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc95dd-9010-4aa8-ba59-7b63d28aba1d service glance] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:15.032122 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc95dd-9010-4aa8-ba59-7b63d28aba1d service glance] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:15.032394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc95dd-9010-4aa8-ba59-7b63d28aba1d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:15.044772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc95dd-9010-4aa8-ba59-7b63d28aba1d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:15.046317 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc95dd-9010-4aa8-ba59-7b63d28aba1d service glance] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:15.057646 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90629e1-1200-4fc7-b3ea-898f723402ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:15.059858 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1026/2070] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:15.060955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90629e1-1200-4fc7-b3ea-898f723402ce service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:15.086592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1045/2071] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:14 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:39:15.208889 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:15.327210 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc95dd-9010-4aa8-ba59-7b63d28aba1d service glance] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:15.329069 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-30dc95dd-9010-4aa8-ba59-7b63d28aba1d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:15.420647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90629e1-1200-4fc7-b3ea-898f723402ce service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:15.435134 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90629e1-1200-4fc7-b3ea-898f723402ce service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:15.446580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e7ee6-ca63-475c-a849-5a9ab4fb935f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:15.447243 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e7ee6-ca63-475c-a849-5a9ab4fb935f service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:15.447523 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e7ee6-ca63-475c-a849-5a9ab4fb935f service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:15.447796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e7ee6-ca63-475c-a849-5a9ab4fb935f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:15.450433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-382e7ee6-ca63-475c-a849-5a9ab4fb935f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:15.500651 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc95dd-9010-4aa8-ba59-7b63d28aba1d service glance] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:15.509299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5995bb-e7c7-4a81-becf-d25bea4580e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:15.510246 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5995bb-e7c7-4a81-becf-d25bea4580e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:15.510581 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5995bb-e7c7-4a81-becf-d25bea4580e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:15.510863 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5995bb-e7c7-4a81-becf-d25bea4580e2 admin admin] PATH_INFO: `/v3/projects/ac240114eedc4eae8ff8d99267f05601` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:15.512191 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5995bb-e7c7-4a81-becf-d25bea4580e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac240114eedc4eae8ff8d99267f05601)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:15.514246 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5995bb-e7c7-4a81-becf-d25bea4580e2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:15.550348 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e7ee6-ca63-475c-a849-5a9ab4fb935f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:15.551941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e7ee6-ca63-475c-a849-5a9ab4fb935f service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:15.569281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1046/2072] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:15.598047 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca4586-33e7-4dce-9b4b-c74f6545d71b None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:15.598334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca4586-33e7-4dce-9b4b-c74f6545d71b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:15.598591 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca4586-33e7-4dce-9b4b-c74f6545d71b None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:15.601165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1047/2073] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jun 7 08:39:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:39:15.610768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fb8ea-9e1c-4aac-9730-0df11df5fe4b None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:15.611253 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fb8ea-9e1c-4aac-9730-0df11df5fe4b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:15.611773 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fb8ea-9e1c-4aac-9730-0df11df5fe4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:15.634752 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9b0fb8ea-9e1c-4aac-9730-0df11df5fe4b None None] MFA Rules not processed for user `1fc0d7bd7e4547f4a8d90ffc2c2e1d03`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:15.636410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9b0fb8ea-9e1c-4aac-9730-0df11df5fe4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:15.668400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:15.686566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c5995bb-e7c7-4a81-becf-d25bea4580e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac240114eedc4eae8ff8d99267f05601', 'request_id': 'req-9c5995bb-e7c7-4a81-becf-d25bea4580e2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:15.692630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c5995bb-e7c7-4a81-becf-d25bea4580e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac240114eedc4eae8ff8d99267f05601', 'request_id': 'req-9c5995bb-e7c7-4a81-becf-d25bea4580e2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:15.699288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c5995bb-e7c7-4a81-becf-d25bea4580e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac240114eedc4eae8ff8d99267f05601', 'request_id': 'req-9c5995bb-e7c7-4a81-becf-d25bea4580e2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:15.716246 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1048/2074] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jun 7 08:39:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:15.727962 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fb8ea-9e1c-4aac-9730-0df11df5fe4b None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:15.729395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9b0fb8ea-9e1c-4aac-9730-0df11df5fe4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:15.748582 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fb8ea-9e1c-4aac-9730-0df11df5fe4b None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:15.816069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae57b3b-bfc4-42dd-bbdb-eec9f21bb5bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc0d7bd7e4547f4a8d90ffc2c2e1d03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:15.817332 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae57b3b-bfc4-42dd-bbdb-eec9f21bb5bf service placement] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:15.817332 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae57b3b-bfc4-42dd-bbdb-eec9f21bb5bf service placement] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:15.817332 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae57b3b-bfc4-42dd-bbdb-eec9f21bb5bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:15.829554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c5995bb-e7c7-4a81-becf-d25bea4580e2 admin admin] The token cache is being invalidate because project ac240114eedc4eae8ff8d99267f05601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:15.830181 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c5995bb-e7c7-4a81-becf-d25bea4580e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:15.830686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae57b3b-bfc4-42dd-bbdb-eec9f21bb5bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:15.831010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c5995bb-e7c7-4a81-becf-d25bea4580e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:15.832225 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae57b3b-bfc4-42dd-bbdb-eec9f21bb5bf service placement] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:15.837928 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1027/2075] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:15 2022] DELETE /identity/v3/projects/ac240114eedc4eae8ff8d99267f05601 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:15.839161 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5995bb-e7c7-4a81-becf-d25bea4580e2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:15.840757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9c5995bb-e7c7-4a81-becf-d25bea4580e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:15.843956 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1049/2076] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:39:15.845286 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56af41-8d49-4f7b-b2c7-bbc4fc5d8c52 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:15.845824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56af41-8d49-4f7b-b2c7-bbc4fc5d8c52 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:15.845824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56af41-8d49-4f7b-b2c7-bbc4fc5d8c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:15.874580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ed56af41-8d49-4f7b-b2c7-bbc4fc5d8c52 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:15.875813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ed56af41-8d49-4f7b-b2c7-bbc4fc5d8c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:15.926757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5995bb-e7c7-4a81-becf-d25bea4580e2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:15.933618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd826ab8-9db1-406d-bc98-4b4a57cdd29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:15.934508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd826ab8-9db1-406d-bc98-4b4a57cdd29d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:15.935066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd826ab8-9db1-406d-bc98-4b4a57cdd29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:15.935066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd826ab8-9db1-406d-bc98-4b4a57cdd29d admin admin] PATH_INFO: `/v3/users/ae58168dadc845daa0958cf975a41abd` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:15.936261 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd826ab8-9db1-406d-bc98-4b4a57cdd29d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae58168dadc845daa0958cf975a41abd)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:15.938484 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd826ab8-9db1-406d-bc98-4b4a57cdd29d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:15.943673 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1050/2077] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:15.952313 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bad70-3aac-4215-a4fa-c48ea62053bb None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:15.956809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bad70-3aac-4215-a4fa-c48ea62053bb None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:15.956809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bad70-3aac-4215-a4fa-c48ea62053bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:15.970885 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-633bad70-3aac-4215-a4fa-c48ea62053bb None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:15.972097 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-633bad70-3aac-4215-a4fa-c48ea62053bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:15.981754 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1051/2078] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:15.985894 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bad70-3aac-4215-a4fa-c48ea62053bb None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:15.997723 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bad70-3aac-4215-a4fa-c48ea62053bb None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:16.002837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd2a65-4f20-48c0-a6db-b7ed531b9c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:16.003546 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd2a65-4f20-48c0-a6db-b7ed531b9c86 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:16.003828 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd2a65-4f20-48c0-a6db-b7ed531b9c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:16.004103 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd2a65-4f20-48c0-a6db-b7ed531b9c86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:16.004828 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd2a65-4f20-48c0-a6db-b7ed531b9c86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:39:16.005170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd2a65-4f20-48c0-a6db-b7ed531b9c86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:16.006933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd2a65-4f20-48c0-a6db-b7ed531b9c86 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:16.019987 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1052/2079] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:39:16.026136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd2a65-4f20-48c0-a6db-b7ed531b9c86 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:16.038136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd2a65-4f20-48c0-a6db-b7ed531b9c86 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:16.043111 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b616b-2924-4c6e-a424-26cb63f3ab11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:16.043772 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b616b-2924-4c6e-a424-26cb63f3ab11 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:16.044057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b616b-2924-4c6e-a424-26cb63f3ab11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:16.044331 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b616b-2924-4c6e-a424-26cb63f3ab11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:16.045101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b616b-2924-4c6e-a424-26cb63f3ab11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:16.047074 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b616b-2924-4c6e-a424-26cb63f3ab11 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:16.135911 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1053/2080] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:16 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:16.140064 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b616b-2924-4c6e-a424-26cb63f3ab11 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:16.143288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dd826ab8-9db1-406d-bc98-4b4a57cdd29d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae58168dadc845daa0958cf975a41abd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:16.144360 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dd826ab8-9db1-406d-bc98-4b4a57cdd29d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae58168dadc845daa0958cf975a41abd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:16.159022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dd826ab8-9db1-406d-bc98-4b4a57cdd29d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae58168dadc845daa0958cf975a41abd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:16.160097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dd826ab8-9db1-406d-bc98-4b4a57cdd29d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae58168dadc845daa0958cf975a41abd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:16.174148 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dd826ab8-9db1-406d-bc98-4b4a57cdd29d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae58168dadc845daa0958cf975a41abd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:16.220660 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b616b-2924-4c6e-a424-26cb63f3ab11 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:16.226129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07adab5-d894-44a8-ba32-a04090ffb34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:16.226804 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07adab5-d894-44a8-ba32-a04090ffb34a admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:16.227091 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07adab5-d894-44a8-ba32-a04090ffb34a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:16.227362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07adab5-d894-44a8-ba32-a04090ffb34a admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:16.228064 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07adab5-d894-44a8-ba32-a04090ffb34a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:16.230152 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07adab5-d894-44a8-ba32-a04090ffb34a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:16.297317 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1028/2081] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:15 2022] DELETE /identity/v3/users/ae58168dadc845daa0958cf975a41abd => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:16.362250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1054/2082] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:16 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:16.366683 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd826ab8-9db1-406d-bc98-4b4a57cdd29d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:16.368221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dd826ab8-9db1-406d-bc98-4b4a57cdd29d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:16.504714 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd826ab8-9db1-406d-bc98-4b4a57cdd29d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:16.517944 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790a361-636a-414f-8ebc-bb9c6b403a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:16.518748 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790a361-636a-414f-8ebc-bb9c6b403a51 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:16.519152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790a361-636a-414f-8ebc-bb9c6b403a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:16.519500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790a361-636a-414f-8ebc-bb9c6b403a51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:16.520346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790a361-636a-414f-8ebc-bb9c6b403a51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:16.522708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790a361-636a-414f-8ebc-bb9c6b403a51 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:16.542523 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1029/2083] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:16.546224 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07adab5-d894-44a8-ba32-a04090ffb34a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:16.562943 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07adab5-d894-44a8-ba32-a04090ffb34a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:16.569691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc6307-9974-4790-a903-c4e896d3856b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:16.570351 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc6307-9974-4790-a903-c4e896d3856b admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:16.570634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc6307-9974-4790-a903-c4e896d3856b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:16.571035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc6307-9974-4790-a903-c4e896d3856b admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:16.571907 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc6307-9974-4790-a903-c4e896d3856b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:16.574028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc6307-9974-4790-a903-c4e896d3856b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:16.590088 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1055/2084] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:39:16.591586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790a361-636a-414f-8ebc-bb9c6b403a51 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:16.607590 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790a361-636a-414f-8ebc-bb9c6b403a51 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:16.610785 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000e49a-f1ea-4685-ad83-1fe7433734d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:16.612142 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000e49a-f1ea-4685-ad83-1fe7433734d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:16.612432 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000e49a-f1ea-4685-ad83-1fe7433734d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:16.612891 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000e49a-f1ea-4685-ad83-1fe7433734d5 admin admin] PATH_INFO: `/v3/projects/c69c83c2a4ab483ab7ebd80bc20b5adb/users/d042e3639da2433398cab53458ee3560/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:16.635660 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000e49a-f1ea-4685-ad83-1fe7433734d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c69c83c2a4ab483ab7ebd80bc20b5adb, user_id=d042e3639da2433398cab53458ee3560, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:16.638520 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000e49a-f1ea-4685-ad83-1fe7433734d5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:16.655612 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:16.667793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc6307-9974-4790-a903-c4e896d3856b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:16.670688 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-72fc6307-9974-4790-a903-c4e896d3856b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:16.695612 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc6307-9974-4790-a903-c4e896d3856b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:16.704391 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6667c3-1c7a-4724-a658-76ec1f660da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:16.705437 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6667c3-1c7a-4724-a658-76ec1f660da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:16.710793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6667c3-1c7a-4724-a658-76ec1f660da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:16.710793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6667c3-1c7a-4724-a658-76ec1f660da3 admin admin] PATH_INFO: `/v3/projects/f26ee9298ae04230979b6a62d24be0e1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:16.710793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6667c3-1c7a-4724-a658-76ec1f660da3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f26ee9298ae04230979b6a62d24be0e1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:16.710793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6667c3-1c7a-4724-a658-76ec1f660da3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:16.730299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1030/2085] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:16 2022] PUT /identity/v3/projects/c69c83c2a4ab483ab7ebd80bc20b5adb/users/d042e3639da2433398cab53458ee3560/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:16.742823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226b5a2-f503-4382-b7b3-5e27dd9c553b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:16.743116 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226b5a2-f503-4382-b7b3-5e27dd9c553b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:16.743426 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226b5a2-f503-4382-b7b3-5e27dd9c553b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:16.769971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=88,keystone:INSERT=7,keystone:DELETE=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:16.779975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b226b5a2-f503-4382-b7b3-5e27dd9c553b None None] MFA Rules not processed for user `d042e3639da2433398cab53458ee3560`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:16.781374 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b226b5a2-f503-4382-b7b3-5e27dd9c553b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:16.825295 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5b6667c3-1c7a-4724-a658-76ec1f660da3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f26ee9298ae04230979b6a62d24be0e1', 'request_id': 'req-5b6667c3-1c7a-4724-a658-76ec1f660da3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:16.832055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5b6667c3-1c7a-4724-a658-76ec1f660da3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f26ee9298ae04230979b6a62d24be0e1', 'request_id': 'req-5b6667c3-1c7a-4724-a658-76ec1f660da3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:16.865255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1031/2086] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:16.869957 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b226b5a2-f503-4382-b7b3-5e27dd9c553b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:16.953722 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b226b5a2-f503-4382-b7b3-5e27dd9c553b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:16.974134 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5b6667c3-1c7a-4724-a658-76ec1f660da3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f26ee9298ae04230979b6a62d24be0e1', 'request_id': 'req-5b6667c3-1c7a-4724-a658-76ec1f660da3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:16.988509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5b6667c3-1c7a-4724-a658-76ec1f660da3 admin admin] The token cache is being invalidate because project f26ee9298ae04230979b6a62d24be0e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:16.989274 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5b6667c3-1c7a-4724-a658-76ec1f660da3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:16.990654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5b6667c3-1c7a-4724-a658-76ec1f660da3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:16.993112 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1056/2087] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:16 2022] DELETE /identity/v3/projects/f26ee9298ae04230979b6a62d24be0e1 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:16.998756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8511f04-5e6f-4873-846a-99e2fecb0fca None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:16.999131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8511f04-5e6f-4873-846a-99e2fecb0fca None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:16.999476 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8511f04-5e6f-4873-846a-99e2fecb0fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:17.018477 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-f8511f04-5e6f-4873-846a-99e2fecb0fca None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:17.021314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f8511f04-5e6f-4873-846a-99e2fecb0fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:17.029697 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c2cac-cc06-4961-8062-684e04a4cec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:17.030899 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c2cac-cc06-4961-8062-684e04a4cec2 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:17.031453 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c2cac-cc06-4961-8062-684e04a4cec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:17.031995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c2cac-cc06-4961-8062-684e04a4cec2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:17.033361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75c2cac-cc06-4961-8062-684e04a4cec2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:17.037724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75c2cac-cc06-4961-8062-684e04a4cec2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:17.085262 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1057/2088] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:17.093832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee40ff-74e0-4533-9cf8-a9fbd31132ac None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:17.094167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee40ff-74e0-4533-9cf8-a9fbd31132ac None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:17.094424 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee40ff-74e0-4533-9cf8-a9fbd31132ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:17.113544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-eaee40ff-74e0-4533-9cf8-a9fbd31132ac None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:17.114742 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-eaee40ff-74e0-4533-9cf8-a9fbd31132ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:17.128662 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1058/2089] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:17.131939 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee40ff-74e0-4533-9cf8-a9fbd31132ac None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:17.141530 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee40ff-74e0-4533-9cf8-a9fbd31132ac None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:17.149417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc82bd-e8c7-42ef-aa83-5332f59ad18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:17.150130 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc82bd-e8c7-42ef-aa83-5332f59ad18f admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:17.150389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc82bd-e8c7-42ef-aa83-5332f59ad18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:17.151512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc82bd-e8c7-42ef-aa83-5332f59ad18f admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:17.151512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cc82bd-e8c7-42ef-aa83-5332f59ad18f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:39:17.151701 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cc82bd-e8c7-42ef-aa83-5332f59ad18f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:17.153395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cc82bd-e8c7-42ef-aa83-5332f59ad18f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:17.164048 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1032/2090] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:16 2022] POST /identity/v3/projects => generated 385 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:17.168264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c2cac-cc06-4961-8062-684e04a4cec2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:17.170172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b75c2cac-cc06-4961-8062-684e04a4cec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:17.175878 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1059/2091] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:39:17.176950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc82bd-e8c7-42ef-aa83-5332f59ad18f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:17.187869 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc82bd-e8c7-42ef-aa83-5332f59ad18f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:17.276489 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c2cac-cc06-4961-8062-684e04a4cec2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:17.282380 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c06f72-6350-49d0-95d0-e96179e4a63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:17.284108 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c06f72-6350-49d0-95d0-e96179e4a63f admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:17.284401 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c06f72-6350-49d0-95d0-e96179e4a63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:17.285035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c06f72-6350-49d0-95d0-e96179e4a63f admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:17.286013 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c06f72-6350-49d0-95d0-e96179e4a63f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:17.288242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4692d-9573-477c-b447-5b02de2d858e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:17.289062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4692d-9573-477c-b447-5b02de2d858e admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:17.289376 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4692d-9573-477c-b447-5b02de2d858e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:17.289486 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c06f72-6350-49d0-95d0-e96179e4a63f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:17.289650 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4692d-9573-477c-b447-5b02de2d858e admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:17.290957 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c4692d-9573-477c-b447-5b02de2d858e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:17.293654 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c4692d-9573-477c-b447-5b02de2d858e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:17.406847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1033/2092] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:17 2022] POST /identity/v3/users => generated 299 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:17.411119 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4692d-9573-477c-b447-5b02de2d858e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:17.411276 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1060/2093] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:17 2022] POST /identity/v3/projects => generated 375 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:17.416081 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c06f72-6350-49d0-95d0-e96179e4a63f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:17.424178 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4692d-9573-477c-b447-5b02de2d858e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:17.491054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c06f72-6350-49d0-95d0-e96179e4a63f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:17.492184 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484b585-3579-4eff-b1ed-34d433500d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:17.493026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484b585-3579-4eff-b1ed-34d433500d7b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:17.493273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484b585-3579-4eff-b1ed-34d433500d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:17.493585 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484b585-3579-4eff-b1ed-34d433500d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:17.495023 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8484b585-3579-4eff-b1ed-34d433500d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:17.496853 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff3bc7-7f6d-4c34-b0e7-3a63f9140b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:17.497672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff3bc7-7f6d-4c34-b0e7-3a63f9140b12 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:17.497995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff3bc7-7f6d-4c34-b0e7-3a63f9140b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:17.498314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff3bc7-7f6d-4c34-b0e7-3a63f9140b12 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:17.499003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8484b585-3579-4eff-b1ed-34d433500d7b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:17.500857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff3bc7-7f6d-4c34-b0e7-3a63f9140b12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:17.502345 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff3bc7-7f6d-4c34-b0e7-3a63f9140b12 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:17.512419 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1034/2094] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:17.515740 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484b585-3579-4eff-b1ed-34d433500d7b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:17.527824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484b585-3579-4eff-b1ed-34d433500d7b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:17.532879 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332dd491-9c0c-465b-97e3-d95fe4714614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:17.534704 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332dd491-9c0c-465b-97e3-d95fe4714614 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:17.537180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332dd491-9c0c-465b-97e3-d95fe4714614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:17.537568 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332dd491-9c0c-465b-97e3-d95fe4714614 admin admin] PATH_INFO: `/v3/projects/5cae3591d93641a0aa81956e67c8fff3/users/c0af89a7707b43398ea7746ac8fbcdc1/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:17.560207 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332dd491-9c0c-465b-97e3-d95fe4714614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cae3591d93641a0aa81956e67c8fff3, user_id=c0af89a7707b43398ea7746ac8fbcdc1, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:17.562979 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332dd491-9c0c-465b-97e3-d95fe4714614 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:17.632227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1061/2095] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:17 2022] POST /identity/v3/users => generated 288 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:17.638933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff3bc7-7f6d-4c34-b0e7-3a63f9140b12 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:17.649822 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff3bc7-7f6d-4c34-b0e7-3a63f9140b12 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:17.654706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2384ddd-c8a4-46f8-9278-a25e09c6c0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:17.655371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2384ddd-c8a4-46f8-9278-a25e09c6c0ba admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:17.655658 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2384ddd-c8a4-46f8-9278-a25e09c6c0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:17.655936 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2384ddd-c8a4-46f8-9278-a25e09c6c0ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:17.656669 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2384ddd-c8a4-46f8-9278-a25e09c6c0ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:17.658387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2384ddd-c8a4-46f8-9278-a25e09c6c0ba admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:17.668858 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1062/2096] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:39:17.672961 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2384ddd-c8a4-46f8-9278-a25e09c6c0ba admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:17.683160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2384ddd-c8a4-46f8-9278-a25e09c6c0ba admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:17.689259 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1035/2097] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:17 2022] PUT /identity/v3/projects/5cae3591d93641a0aa81956e67c8fff3/users/c0af89a7707b43398ea7746ac8fbcdc1/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:39:17.695554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332dd491-9c0c-465b-97e3-d95fe4714614 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:17.748753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4561903-b401-4277-ac44-ff5446608a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:17.749370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4561903-b401-4277-ac44-ff5446608a7b admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:17.749700 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4561903-b401-4277-ac44-ff5446608a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:17.750103 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4561903-b401-4277-ac44-ff5446608a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:17.750848 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4561903-b401-4277-ac44-ff5446608a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:17.752533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4561903-b401-4277-ac44-ff5446608a7b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:17.768568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1063/2098] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:39:17.770068 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4561903-b401-4277-ac44-ff5446608a7b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:17.772483 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332dd491-9c0c-465b-97e3-d95fe4714614 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:17.778398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77739c2-5d8f-471e-8150-549c37ecc767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:17.779511 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77739c2-5d8f-471e-8150-549c37ecc767 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:17.779897 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77739c2-5d8f-471e-8150-549c37ecc767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:17.780273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77739c2-5d8f-471e-8150-549c37ecc767 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:17.781353 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77739c2-5d8f-471e-8150-549c37ecc767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:17.782078 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4561903-b401-4277-ac44-ff5446608a7b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:17.783431 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77739c2-5d8f-471e-8150-549c37ecc767 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:17.789398 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ec9b4-7a1b-49ea-adae-2b94f474d639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:17.790800 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ec9b4-7a1b-49ea-adae-2b94f474d639 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:17.791297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ec9b4-7a1b-49ea-adae-2b94f474d639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:17.791752 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ec9b4-7a1b-49ea-adae-2b94f474d639 admin admin] PATH_INFO: `/v3/projects/5daf862eb4de4ea992b35692f003a25e/users/4e5893f0909446ff82b5a72584ecae37/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:17.795532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1036/2099] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:17.801983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77739c2-5d8f-471e-8150-549c37ecc767 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:17.813144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ec9b4-7a1b-49ea-adae-2b94f474d639 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5daf862eb4de4ea992b35692f003a25e, user_id=4e5893f0909446ff82b5a72584ecae37, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:17.813355 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77739c2-5d8f-471e-8150-549c37ecc767 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:17.815824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ec9b4-7a1b-49ea-adae-2b94f474d639 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:17.819208 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a4ed1-28cf-4630-bbf6-779ea0cd1be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:17.820150 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a4ed1-28cf-4630-bbf6-779ea0cd1be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:17.820445 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a4ed1-28cf-4630-bbf6-779ea0cd1be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:17.820756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a4ed1-28cf-4630-bbf6-779ea0cd1be7 admin admin] PATH_INFO: `/v3/projects/5cae3591d93641a0aa81956e67c8fff3/users/c0af89a7707b43398ea7746ac8fbcdc1/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:17.822806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a4ed1-28cf-4630-bbf6-779ea0cd1be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cae3591d93641a0aa81956e67c8fff3, user_id=c0af89a7707b43398ea7746ac8fbcdc1, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:17.825609 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a4ed1-28cf-4630-bbf6-779ea0cd1be7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:17.836892 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1037/2100] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:17 2022] PUT /identity/v3/projects/5cae3591d93641a0aa81956e67c8fff3/users/c0af89a7707b43398ea7746ac8fbcdc1/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:17.847302 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e2204-29df-40a1-b086-58ade46b582e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:17.847584 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e2204-29df-40a1-b086-58ade46b582e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:17.847928 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e2204-29df-40a1-b086-58ade46b582e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:17.877230 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-404e2204-29df-40a1-b086-58ade46b582e None None] MFA Rules not processed for user `c0af89a7707b43398ea7746ac8fbcdc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:17.878808 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-404e2204-29df-40a1-b086-58ade46b582e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:17.899189 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1064/2101] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:17 2022] PUT /identity/v3/projects/5daf862eb4de4ea992b35692f003a25e/users/4e5893f0909446ff82b5a72584ecae37/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:17.908254 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a4b98-ecea-4389-81ad-350d9e23baa1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:17.909188 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a4b98-ecea-4389-81ad-350d9e23baa1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:17.909472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a4b98-ecea-4389-81ad-350d9e23baa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:17.931600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-952a4b98-ecea-4389-81ad-350d9e23baa1 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:17.932894 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-952a4b98-ecea-4389-81ad-350d9e23baa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:17.944525 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:17.984773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1038/2102] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:17.989455 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e2204-29df-40a1-b086-58ade46b582e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:17.991821 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-404e2204-29df-40a1-b086-58ade46b582e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:18.035978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1065/2103] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:18.035978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a4b98-ecea-4389-81ad-350d9e23baa1 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:18.035978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-952a4b98-ecea-4389-81ad-350d9e23baa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:18.078461 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e2204-29df-40a1-b086-58ade46b582e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:18.084145 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b91341-b3ba-4caa-a36a-a87cc9fd11ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:18.085119 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b91341-b3ba-4caa-a36a-a87cc9fd11ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:18.085402 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b91341-b3ba-4caa-a36a-a87cc9fd11ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:18.085680 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b91341-b3ba-4caa-a36a-a87cc9fd11ec admin admin] PATH_INFO: `/v3/users/2c5023c3a1b0471e961b6ca563c380da` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:18.086958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b91341-b3ba-4caa-a36a-a87cc9fd11ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c5023c3a1b0471e961b6ca563c380da)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:18.088942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b91341-b3ba-4caa-a36a-a87cc9fd11ec admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:18.114107 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a4b98-ecea-4389-81ad-350d9e23baa1 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:18.121685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71f580-1e8f-482a-b634-b9bff79d7c80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:18.122864 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71f580-1e8f-482a-b634-b9bff79d7c80 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:18.123402 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71f580-1e8f-482a-b634-b9bff79d7c80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:18.123885 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71f580-1e8f-482a-b634-b9bff79d7c80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:18.203632 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:18.227006 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac71f580-1e8f-482a-b634-b9bff79d7c80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:18.228530 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac71f580-1e8f-482a-b634-b9bff79d7c80 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:18.239405 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1066/2104] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:18.240860 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71f580-1e8f-482a-b634-b9bff79d7c80 service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:18.242349 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ac71f580-1e8f-482a-b634-b9bff79d7c80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:18.298295 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71f580-1e8f-482a-b634-b9bff79d7c80 service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:18.301931 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cc317-a5a8-4ff2-b11d-7feb031262b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:18.303289 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cc317-a5a8-4ff2-b11d-7feb031262b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:18.303579 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cc317-a5a8-4ff2-b11d-7feb031262b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:18.303935 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cc317-a5a8-4ff2-b11d-7feb031262b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:18.313320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871cc317-a5a8-4ff2-b11d-7feb031262b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:18.314954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871cc317-a5a8-4ff2-b11d-7feb031262b6 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:18.325046 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1067/2105] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:18.357131 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-08b91341-b3ba-4caa-a36a-a87cc9fd11ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c5023c3a1b0471e961b6ca563c380da'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:18.357922 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-08b91341-b3ba-4caa-a36a-a87cc9fd11ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c5023c3a1b0471e961b6ca563c380da'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:18.371071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-08b91341-b3ba-4caa-a36a-a87cc9fd11ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c5023c3a1b0471e961b6ca563c380da'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:18.371832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-08b91341-b3ba-4caa-a36a-a87cc9fd11ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c5023c3a1b0471e961b6ca563c380da'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:18.385907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-08b91341-b3ba-4caa-a36a-a87cc9fd11ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c5023c3a1b0471e961b6ca563c380da'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:18.461224 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1039/2106] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:17 2022] DELETE /identity/v3/users/2c5023c3a1b0471e961b6ca563c380da => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:18.463560 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df3bb5-ea5b-46a4-836e-452b3911c68f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:18.463849 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df3bb5-ea5b-46a4-836e-452b3911c68f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:18.464127 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df3bb5-ea5b-46a4-836e-452b3911c68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:18.497348 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-47df3bb5-ea5b-46a4-836e-452b3911c68f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:18.498960 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-47df3bb5-ea5b-46a4-836e-452b3911c68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:18.565187 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1068/2107] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:18.575277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b91341-b3ba-4caa-a36a-a87cc9fd11ec admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:18.576703 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-08b91341-b3ba-4caa-a36a-a87cc9fd11ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:18.649995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b91341-b3ba-4caa-a36a-a87cc9fd11ec admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:18.661529 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f14ff7-8d9e-49fd-a102-c81f060e997f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:18.663565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f14ff7-8d9e-49fd-a102-c81f060e997f service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:18.663931 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f14ff7-8d9e-49fd-a102-c81f060e997f service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:18.664345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f14ff7-8d9e-49fd-a102-c81f060e997f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:18.677390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f14ff7-8d9e-49fd-a102-c81f060e997f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:18.679523 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f14ff7-8d9e-49fd-a102-c81f060e997f service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:18.691681 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1040/2108] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:18.705888 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:18.995173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:19.376464 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df3bb5-ea5b-46a4-836e-452b3911c68f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:19.378052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-47df3bb5-ea5b-46a4-836e-452b3911c68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:19.393389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df3bb5-ea5b-46a4-836e-452b3911c68f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:19.399146 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024d651-1c1c-41a3-b87c-ffff8c01f6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:19.400012 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024d651-1c1c-41a3-b87c-ffff8c01f6f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:19.400295 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024d651-1c1c-41a3-b87c-ffff8c01f6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:19.400572 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024d651-1c1c-41a3-b87c-ffff8c01f6f0 admin admin] PATH_INFO: `/v3/projects/a2b6cf5dc2584b8aa03904869a1ef92e` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:19.401932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024d651-1c1c-41a3-b87c-ffff8c01f6f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2b6cf5dc2584b8aa03904869a1ef92e)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:19.404237 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024d651-1c1c-41a3-b87c-ffff8c01f6f0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:19.551740 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6024d651-1c1c-41a3-b87c-ffff8c01f6f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2b6cf5dc2584b8aa03904869a1ef92e', 'request_id': 'req-6024d651-1c1c-41a3-b87c-ffff8c01f6f0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:19.558476 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6024d651-1c1c-41a3-b87c-ffff8c01f6f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2b6cf5dc2584b8aa03904869a1ef92e', 'request_id': 'req-6024d651-1c1c-41a3-b87c-ffff8c01f6f0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:19.625781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6024d651-1c1c-41a3-b87c-ffff8c01f6f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2b6cf5dc2584b8aa03904869a1ef92e', 'request_id': 'req-6024d651-1c1c-41a3-b87c-ffff8c01f6f0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:19.640285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6024d651-1c1c-41a3-b87c-ffff8c01f6f0 admin admin] The token cache is being invalidate because project a2b6cf5dc2584b8aa03904869a1ef92e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:19.640714 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6024d651-1c1c-41a3-b87c-ffff8c01f6f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:19.641445 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6024d651-1c1c-41a3-b87c-ffff8c01f6f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:19.643508 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1069/2109] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:19 2022] DELETE /identity/v3/projects/a2b6cf5dc2584b8aa03904869a1ef92e => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:19.647662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f14ff7-8d9e-49fd-a102-c81f060e997f service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:19.662684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f14ff7-8d9e-49fd-a102-c81f060e997f service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:19.711337 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024d651-1c1c-41a3-b87c-ffff8c01f6f0 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:19.714832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6024d651-1c1c-41a3-b87c-ffff8c01f6f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:19.751552 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9cb014-eabc-4892-be74-fddc1e5631f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:19.751785 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cb014-eabc-4892-be74-fddc1e5631f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:19.752057 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cb014-eabc-4892-be74-fddc1e5631f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:19.752330 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cb014-eabc-4892-be74-fddc1e5631f6 admin admin] PATH_INFO: `/v3/users/4e61b774e49c47d5adf60d422c9d0970` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:19.753999 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9cb014-eabc-4892-be74-fddc1e5631f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e61b774e49c47d5adf60d422c9d0970)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:19.756346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9cb014-eabc-4892-be74-fddc1e5631f6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:19.829392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024d651-1c1c-41a3-b87c-ffff8c01f6f0 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:19.834446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7c181-aa0d-4eb7-b000-b10c2f79bf6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:19.835425 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7c181-aa0d-4eb7-b000-b10c2f79bf6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:19.835705 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7c181-aa0d-4eb7-b000-b10c2f79bf6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:19.835977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7c181-aa0d-4eb7-b000-b10c2f79bf6c admin admin] PATH_INFO: `/v3/users/d042e3639da2433398cab53458ee3560` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:19.837428 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7c181-aa0d-4eb7-b000-b10c2f79bf6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d042e3639da2433398cab53458ee3560)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:19.839368 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7c181-aa0d-4eb7-b000-b10c2f79bf6c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:19.914716 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:19.980802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7c9cb014-eabc-4892-be74-fddc1e5631f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e61b774e49c47d5adf60d422c9d0970'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:19.981777 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7c9cb014-eabc-4892-be74-fddc1e5631f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e61b774e49c47d5adf60d422c9d0970'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:19.992314 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7c9cb014-eabc-4892-be74-fddc1e5631f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e61b774e49c47d5adf60d422c9d0970'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:19.993388 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7c9cb014-eabc-4892-be74-fddc1e5631f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e61b774e49c47d5adf60d422c9d0970'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:20.016408 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7c9cb014-eabc-4892-be74-fddc1e5631f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e61b774e49c47d5adf60d422c9d0970'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:20.038066 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0ae7c181-aa0d-4eb7-b000-b10c2f79bf6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd042e3639da2433398cab53458ee3560'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:20.039080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0ae7c181-aa0d-4eb7-b000-b10c2f79bf6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd042e3639da2433398cab53458ee3560'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:20.046739 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0ae7c181-aa0d-4eb7-b000-b10c2f79bf6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd042e3639da2433398cab53458ee3560'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:20.047545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0ae7c181-aa0d-4eb7-b000-b10c2f79bf6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd042e3639da2433398cab53458ee3560'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:20.154002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1041/2110] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:19 2022] DELETE /identity/v3/users/4e61b774e49c47d5adf60d422c9d0970 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:20.253931 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0ae7c181-aa0d-4eb7-b000-b10c2f79bf6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd042e3639da2433398cab53458ee3560'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:20.295940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1070/2111] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:19 2022] DELETE /identity/v3/users/d042e3639da2433398cab53458ee3560 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:39:20.301266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9625e-f298-41cd-a356-73826de568a6 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:20.301554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9625e-f298-41cd-a356-73826de568a6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:20.301834 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9625e-f298-41cd-a356-73826de568a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:20.321137 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c6d9625e-f298-41cd-a356-73826de568a6 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:20.322601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c6d9625e-f298-41cd-a356-73826de568a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:20.340070 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:20.421425 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1042/2112] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:20.445168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7c181-aa0d-4eb7-b000-b10c2f79bf6c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:20.446783 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0ae7c181-aa0d-4eb7-b000-b10c2f79bf6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:20.527175 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7c181-aa0d-4eb7-b000-b10c2f79bf6c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:20.532362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4fe91-60e9-4171-b4d2-dcda961ba2fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:20.533026 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4fe91-60e9-4171-b4d2-dcda961ba2fb service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:20.533316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4fe91-60e9-4171-b4d2-dcda961ba2fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:20.533594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4fe91-60e9-4171-b4d2-dcda961ba2fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:20.545534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4fe91-60e9-4171-b4d2-dcda961ba2fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:20.547041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4fe91-60e9-4171-b4d2-dcda961ba2fb service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:20.585943 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1071/2113] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:20.768990 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9625e-f298-41cd-a356-73826de568a6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:20.772417 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c6d9625e-f298-41cd-a356-73826de568a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:20.832575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9625e-f298-41cd-a356-73826de568a6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:20.845723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd1d56-ca6d-4183-b40d-e79200b4f2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:20.849150 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd1d56-ca6d-4183-b40d-e79200b4f2c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:20.849535 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd1d56-ca6d-4183-b40d-e79200b4f2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:20.849820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd1d56-ca6d-4183-b40d-e79200b4f2c2 admin admin] PATH_INFO: `/v3/projects/993055fe255948b38a39bd8e79bb1ad8` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:20.855611 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd1d56-ca6d-4183-b40d-e79200b4f2c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=993055fe255948b38a39bd8e79bb1ad8)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:20.859128 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd1d56-ca6d-4183-b40d-e79200b4f2c2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:21.057683 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-21fd1d56-ca6d-4183-b40d-e79200b4f2c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '993055fe255948b38a39bd8e79bb1ad8', 'request_id': 'req-21fd1d56-ca6d-4183-b40d-e79200b4f2c2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:21.065361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-21fd1d56-ca6d-4183-b40d-e79200b4f2c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '993055fe255948b38a39bd8e79bb1ad8', 'request_id': 'req-21fd1d56-ca6d-4183-b40d-e79200b4f2c2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:21.073312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-21fd1d56-ca6d-4183-b40d-e79200b4f2c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '993055fe255948b38a39bd8e79bb1ad8', 'request_id': 'req-21fd1d56-ca6d-4183-b40d-e79200b4f2c2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:21.170903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-21fd1d56-ca6d-4183-b40d-e79200b4f2c2 admin admin] The token cache is being invalidate because project 993055fe255948b38a39bd8e79bb1ad8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:21.170903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-21fd1d56-ca6d-4183-b40d-e79200b4f2c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:21.170903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-21fd1d56-ca6d-4183-b40d-e79200b4f2c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:21.171789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1043/2114] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:20 2022] DELETE /identity/v3/projects/993055fe255948b38a39bd8e79bb1ad8 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:21.176823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf012c24-0f4e-4a41-b26d-c7ddef1b1e76 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:21.177189 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf012c24-0f4e-4a41-b26d-c7ddef1b1e76 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:21.178056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf012c24-0f4e-4a41-b26d-c7ddef1b1e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:21.205503 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-bf012c24-0f4e-4a41-b26d-c7ddef1b1e76 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:21.206954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-bf012c24-0f4e-4a41-b26d-c7ddef1b1e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:21.306382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1072/2115] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:21.314245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96369538-dab1-4b3a-bb5f-b1ecc6b4223d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:21.314534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96369538-dab1-4b3a-bb5f-b1ecc6b4223d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:21.314814 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96369538-dab1-4b3a-bb5f-b1ecc6b4223d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:21.348871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-96369538-dab1-4b3a-bb5f-b1ecc6b4223d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:21.349107 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf012c24-0f4e-4a41-b26d-c7ddef1b1e76 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:21.355454 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-bf012c24-0f4e-4a41-b26d-c7ddef1b1e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:21.384734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-96369538-dab1-4b3a-bb5f-b1ecc6b4223d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:21.402019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf012c24-0f4e-4a41-b26d-c7ddef1b1e76 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:21.414718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1044/2116] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:21.419754 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96369538-dab1-4b3a-bb5f-b1ecc6b4223d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:21.423655 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8aef3c-a3c8-4fdc-b29a-40eb88dc339c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:21.426819 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8aef3c-a3c8-4fdc-b29a-40eb88dc339c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:21.427111 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8aef3c-a3c8-4fdc-b29a-40eb88dc339c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:21.428461 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8aef3c-a3c8-4fdc-b29a-40eb88dc339c admin admin] PATH_INFO: `/v3/projects/c69c83c2a4ab483ab7ebd80bc20b5adb` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:21.432025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8aef3c-a3c8-4fdc-b29a-40eb88dc339c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c69c83c2a4ab483ab7ebd80bc20b5adb)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:21.437383 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8aef3c-a3c8-4fdc-b29a-40eb88dc339c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:21.448244 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96369538-dab1-4b3a-bb5f-b1ecc6b4223d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:21.463673 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9a50a-4ba3-4a06-bf64-2a7173fdcc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:21.468201 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9a50a-4ba3-4a06-bf64-2a7173fdcc0c admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:21.468201 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9a50a-4ba3-4a06-bf64-2a7173fdcc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:21.468201 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9a50a-4ba3-4a06-bf64-2a7173fdcc0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:21.468201 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9a50a-4ba3-4a06-bf64-2a7173fdcc0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:39:21.468201 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9a50a-4ba3-4a06-bf64-2a7173fdcc0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:21.468814 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9a50a-4ba3-4a06-bf64-2a7173fdcc0c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:21.487900 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1045/2117] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:39:21.489864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:21.492421 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9a50a-4ba3-4a06-bf64-2a7173fdcc0c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:21.508211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9a50a-4ba3-4a06-bf64-2a7173fdcc0c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:21.514065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c1119-3ac1-4dfe-b4af-b620b94e47ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:21.514751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c1119-3ac1-4dfe-b4af-b620b94e47ee admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:21.515348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c1119-3ac1-4dfe-b4af-b620b94e47ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:21.515652 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c1119-3ac1-4dfe-b4af-b620b94e47ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:21.516804 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c1119-3ac1-4dfe-b4af-b620b94e47ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:21.518853 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c1119-3ac1-4dfe-b4af-b620b94e47ee admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:21.575366 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7b8aef3c-a3c8-4fdc-b29a-40eb88dc339c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c69c83c2a4ab483ab7ebd80bc20b5adb', 'request_id': 'req-7b8aef3c-a3c8-4fdc-b29a-40eb88dc339c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:21.585569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7b8aef3c-a3c8-4fdc-b29a-40eb88dc339c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c69c83c2a4ab483ab7ebd80bc20b5adb', 'request_id': 'req-7b8aef3c-a3c8-4fdc-b29a-40eb88dc339c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:21.613047 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1046/2118] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:21 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:21.616911 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c1119-3ac1-4dfe-b4af-b620b94e47ee admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:21.642672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7b8aef3c-a3c8-4fdc-b29a-40eb88dc339c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c69c83c2a4ab483ab7ebd80bc20b5adb', 'request_id': 'req-7b8aef3c-a3c8-4fdc-b29a-40eb88dc339c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:21.662764 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7b8aef3c-a3c8-4fdc-b29a-40eb88dc339c admin admin] The token cache is being invalidate because project c69c83c2a4ab483ab7ebd80bc20b5adb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:21.663014 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7b8aef3c-a3c8-4fdc-b29a-40eb88dc339c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:21.663643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7b8aef3c-a3c8-4fdc-b29a-40eb88dc339c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:21.665521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1073/2119] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:21 2022] DELETE /identity/v3/projects/c69c83c2a4ab483ab7ebd80bc20b5adb => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:21.669986 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8aef3c-a3c8-4fdc-b29a-40eb88dc339c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:21.671516 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7b8aef3c-a3c8-4fdc-b29a-40eb88dc339c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:21.693451 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c1119-3ac1-4dfe-b4af-b620b94e47ee admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:21.710206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26953e5-a0a3-4808-a9f0-4da4dd712e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:21.710206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26953e5-a0a3-4808-a9f0-4da4dd712e16 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:21.710206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26953e5-a0a3-4808-a9f0-4da4dd712e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:21.710206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26953e5-a0a3-4808-a9f0-4da4dd712e16 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:21.710667 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26953e5-a0a3-4808-a9f0-4da4dd712e16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:21.712762 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26953e5-a0a3-4808-a9f0-4da4dd712e16 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:21.747607 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8aef3c-a3c8-4fdc-b29a-40eb88dc339c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:21.752597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b45a6e-f64b-4713-8e81-65526c9b8865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:21.753512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b45a6e-f64b-4713-8e81-65526c9b8865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:21.753790 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b45a6e-f64b-4713-8e81-65526c9b8865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:21.754059 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b45a6e-f64b-4713-8e81-65526c9b8865 admin admin] PATH_INFO: `/v3/users/c0af89a7707b43398ea7746ac8fbcdc1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:21.755287 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b45a6e-f64b-4713-8e81-65526c9b8865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0af89a7707b43398ea7746ac8fbcdc1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:21.757297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b45a6e-f64b-4713-8e81-65526c9b8865 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:21.833129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:21.850925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1047/2120] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:21 2022] POST /identity/v3/users => generated 288 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:21.856619 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26953e5-a0a3-4808-a9f0-4da4dd712e16 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:21.858906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c26953e5-a0a3-4808-a9f0-4da4dd712e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:21.882961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26953e5-a0a3-4808-a9f0-4da4dd712e16 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:21.901763 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17427bad-7344-46e5-8ddf-b55c9c9cc08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:21.902807 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17427bad-7344-46e5-8ddf-b55c9c9cc08f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:21.903248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17427bad-7344-46e5-8ddf-b55c9c9cc08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:21.903681 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17427bad-7344-46e5-8ddf-b55c9c9cc08f admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:21.904453 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17427bad-7344-46e5-8ddf-b55c9c9cc08f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:21.906590 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17427bad-7344-46e5-8ddf-b55c9c9cc08f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:21.937820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1048/2121] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:21.950812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17427bad-7344-46e5-8ddf-b55c9c9cc08f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:21.963781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17427bad-7344-46e5-8ddf-b55c9c9cc08f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:21.981586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e285e-02a1-49d3-bd7e-9624134f671b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:21.984075 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e285e-02a1-49d3-bd7e-9624134f671b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:21.984735 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e285e-02a1-49d3-bd7e-9624134f671b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:21.986669 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e285e-02a1-49d3-bd7e-9624134f671b admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:21.987756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e285e-02a1-49d3-bd7e-9624134f671b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:21.991741 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-17b45a6e-f64b-4713-8e81-65526c9b8865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0af89a7707b43398ea7746ac8fbcdc1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:21.992673 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-17b45a6e-f64b-4713-8e81-65526c9b8865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0af89a7707b43398ea7746ac8fbcdc1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:21.993449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e285e-02a1-49d3-bd7e-9624134f671b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:22.014076 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-17b45a6e-f64b-4713-8e81-65526c9b8865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0af89a7707b43398ea7746ac8fbcdc1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:22.015383 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-17b45a6e-f64b-4713-8e81-65526c9b8865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0af89a7707b43398ea7746ac8fbcdc1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:22.022576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1049/2122] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:22.026193 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e285e-02a1-49d3-bd7e-9624134f671b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:22.028410 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e08e285e-02a1-49d3-bd7e-9624134f671b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:22.058705 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e285e-02a1-49d3-bd7e-9624134f671b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:22.064486 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6d149-c249-44e8-8d31-b959ae3250c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:22.065647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6d149-c249-44e8-8d31-b959ae3250c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:22.066144 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6d149-c249-44e8-8d31-b959ae3250c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:22.066422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6d149-c249-44e8-8d31-b959ae3250c5 admin admin] PATH_INFO: `/v3/projects/c0f861e2663f4e14827fb419a947fda7/users/b0c10b946cb94e0f89674cf94e5cbc86/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:22.078292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-17b45a6e-f64b-4713-8e81-65526c9b8865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0af89a7707b43398ea7746ac8fbcdc1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:22.096662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6d149-c249-44e8-8d31-b959ae3250c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0f861e2663f4e14827fb419a947fda7, user_id=b0c10b946cb94e0f89674cf94e5cbc86, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:22.101748 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6d149-c249-44e8-8d31-b959ae3250c5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:22.120497 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1074/2123] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:21 2022] DELETE /identity/v3/users/c0af89a7707b43398ea7746ac8fbcdc1 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:22.234951 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1050/2124] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:22 2022] PUT /identity/v3/projects/c0f861e2663f4e14827fb419a947fda7/users/b0c10b946cb94e0f89674cf94e5cbc86/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:22.236748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c020d-73f8-4740-864e-bfee1a5eace2 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:22.237072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c020d-73f8-4740-864e-bfee1a5eace2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:22.237358 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c020d-73f8-4740-864e-bfee1a5eace2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:22.278348 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-706c020d-73f8-4740-864e-bfee1a5eace2 None None] MFA Rules not processed for user `b0c10b946cb94e0f89674cf94e5cbc86`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:22.279623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-706c020d-73f8-4740-864e-bfee1a5eace2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:22.371229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1075/2125] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:22.376511 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6d149-c249-44e8-8d31-b959ae3250c5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:22.450278 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6d149-c249-44e8-8d31-b959ae3250c5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:22.459623 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59b794-df49-44c8-ac55-967d57f502aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:22.459623 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59b794-df49-44c8-ac55-967d57f502aa admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:22.459623 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59b794-df49-44c8-ac55-967d57f502aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:22.459623 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59b794-df49-44c8-ac55-967d57f502aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:22.460062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59b794-df49-44c8-ac55-967d57f502aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:22.460141 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59b794-df49-44c8-ac55-967d57f502aa admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:22.570751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1051/2126] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:22 2022] POST /identity/v3/projects => generated 377 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:22.574716 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c020d-73f8-4740-864e-bfee1a5eace2 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:22.588790 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c020d-73f8-4740-864e-bfee1a5eace2 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:22.674731 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d5686-c4fe-48c5-aad3-e0d6b4bbc5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:22.675895 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d5686-c4fe-48c5-aad3-e0d6b4bbc5c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:22.676187 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d5686-c4fe-48c5-aad3-e0d6b4bbc5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:22.676470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d5686-c4fe-48c5-aad3-e0d6b4bbc5c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:22.678054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d5686-c4fe-48c5-aad3-e0d6b4bbc5c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:22.680040 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d5686-c4fe-48c5-aad3-e0d6b4bbc5c6 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:22.774097 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1076/2127] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:22 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:22.775424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59b794-df49-44c8-ac55-967d57f502aa admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:22.787931 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59b794-df49-44c8-ac55-967d57f502aa admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:22.795445 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92389ec1-13b3-48a4-949d-f451a8bca290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:22.797539 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92389ec1-13b3-48a4-949d-f451a8bca290 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:22.797945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92389ec1-13b3-48a4-949d-f451a8bca290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:22.798339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92389ec1-13b3-48a4-949d-f451a8bca290 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:22.799273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92389ec1-13b3-48a4-949d-f451a8bca290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:22.801416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92389ec1-13b3-48a4-949d-f451a8bca290 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:22.816286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1052/2128] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:22.820235 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d5686-c4fe-48c5-aad3-e0d6b4bbc5c6 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:22.839127 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d5686-c4fe-48c5-aad3-e0d6b4bbc5c6 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:22.855249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf3f5a-ef4d-424b-81c1-734cd1a67786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:22.855823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf3f5a-ef4d-424b-81c1-734cd1a67786 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:22.856112 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf3f5a-ef4d-424b-81c1-734cd1a67786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:22.856391 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf3f5a-ef4d-424b-81c1-734cd1a67786 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:22.857231 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf3f5a-ef4d-424b-81c1-734cd1a67786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:22.859346 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf3f5a-ef4d-424b-81c1-734cd1a67786 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:22.872658 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1077/2129] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:39:22.877690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92389ec1-13b3-48a4-949d-f451a8bca290 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:22.891936 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92389ec1-13b3-48a4-949d-f451a8bca290 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:22.906037 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436535d-298c-4667-827d-b60259cd3ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:22.907058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436535d-298c-4667-827d-b60259cd3ce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:22.907452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436535d-298c-4667-827d-b60259cd3ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:22.907736 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436535d-298c-4667-827d-b60259cd3ce3 admin admin] PATH_INFO: `/v3/projects/d4c13e578e6e4b2981adafa571b9d197/users/eabc28b6ace5413e882afb8a98d6e8fa/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:22.926359 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436535d-298c-4667-827d-b60259cd3ce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4c13e578e6e4b2981adafa571b9d197, user_id=eabc28b6ace5413e882afb8a98d6e8fa, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:22.929961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436535d-298c-4667-827d-b60259cd3ce3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:22.942146 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf3f5a-ef4d-424b-81c1-734cd1a67786 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:22.944130 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e5bf3f5a-ef4d-424b-81c1-734cd1a67786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:22.970946 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf3f5a-ef4d-424b-81c1-734cd1a67786 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:22.996771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2bb0e-f4e9-42b3-9d88-2c9ec2173d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:22.998137 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2bb0e-f4e9-42b3-9d88-2c9ec2173d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:22.998509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2bb0e-f4e9-42b3-9d88-2c9ec2173d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:22.998717 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2bb0e-f4e9-42b3-9d88-2c9ec2173d20 admin admin] PATH_INFO: `/v3/projects/5cae3591d93641a0aa81956e67c8fff3` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:23.000674 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2bb0e-f4e9-42b3-9d88-2c9ec2173d20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cae3591d93641a0aa81956e67c8fff3)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:23.005717 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2bb0e-f4e9-42b3-9d88-2c9ec2173d20 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:23.051106 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1053/2130] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:22 2022] PUT /identity/v3/projects/d4c13e578e6e4b2981adafa571b9d197/users/eabc28b6ace5413e882afb8a98d6e8fa/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:23.054204 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fda591-0327-48af-907b-de8bd3287da0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:23.054859 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fda591-0327-48af-907b-de8bd3287da0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:23.055203 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fda591-0327-48af-907b-de8bd3287da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:23.061059 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:23.088808 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b7fda591-0327-48af-907b-de8bd3287da0 None None] MFA Rules not processed for user `eabc28b6ace5413e882afb8a98d6e8fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:23.090073 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b7fda591-0327-48af-907b-de8bd3287da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:23.172389 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1054/2131] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:23.178342 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fda591-0327-48af-907b-de8bd3287da0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:23.200587 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9bc2bb0e-f4e9-42b3-9d88-2c9ec2173d20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cae3591d93641a0aa81956e67c8fff3', 'request_id': 'req-9bc2bb0e-f4e9-42b3-9d88-2c9ec2173d20'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:23.208476 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9bc2bb0e-f4e9-42b3-9d88-2c9ec2173d20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cae3591d93641a0aa81956e67c8fff3', 'request_id': 'req-9bc2bb0e-f4e9-42b3-9d88-2c9ec2173d20'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:23.302776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fda591-0327-48af-907b-de8bd3287da0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:23.304793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9bc2bb0e-f4e9-42b3-9d88-2c9ec2173d20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cae3591d93641a0aa81956e67c8fff3', 'request_id': 'req-9bc2bb0e-f4e9-42b3-9d88-2c9ec2173d20'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:23.326782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9bc2bb0e-f4e9-42b3-9d88-2c9ec2173d20 admin admin] The token cache is being invalidate because project 5cae3591d93641a0aa81956e67c8fff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:23.327812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9bc2bb0e-f4e9-42b3-9d88-2c9ec2173d20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:23.327927 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9bc2bb0e-f4e9-42b3-9d88-2c9ec2173d20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:23.329572 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1078/2132] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:22 2022] DELETE /identity/v3/projects/5cae3591d93641a0aa81956e67c8fff3 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:23.336480 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2626ba-9f39-4791-af07-98ee540fd370 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:23.336759 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2626ba-9f39-4791-af07-98ee540fd370 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:23.337068 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2626ba-9f39-4791-af07-98ee540fd370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:23.370351 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b56ba-a2e5-4378-9548-ea5759dc76ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:23.372498 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b56ba-a2e5-4378-9548-ea5759dc76ce admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:23.373024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b56ba-a2e5-4378-9548-ea5759dc76ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:23.373236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-3c2626ba-9f39-4791-af07-98ee540fd370 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:23.373607 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b56ba-a2e5-4378-9548-ea5759dc76ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:23.374473 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b56ba-a2e5-4378-9548-ea5759dc76ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:23.374666 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3c2626ba-9f39-4791-af07-98ee540fd370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:23.377205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b56ba-a2e5-4378-9548-ea5759dc76ce admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:23.385542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1079/2133] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:23.393573 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83ee11-dba9-4b0e-b68f-4aea09239823 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:23.393716 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83ee11-dba9-4b0e-b68f-4aea09239823 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:23.393994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83ee11-dba9-4b0e-b68f-4aea09239823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:23.427599 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7a83ee11-dba9-4b0e-b68f-4aea09239823 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:23.429090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7a83ee11-dba9-4b0e-b68f-4aea09239823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:23.442862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1080/2134] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:23.442862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83ee11-dba9-4b0e-b68f-4aea09239823 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:23.442862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7a83ee11-dba9-4b0e-b68f-4aea09239823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:23.457033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83ee11-dba9-4b0e-b68f-4aea09239823 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:23.472095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ac0b4-07ec-405f-888e-bb41df0bbd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:23.473699 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ac0b4-07ec-405f-888e-bb41df0bbd91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:23.473989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ac0b4-07ec-405f-888e-bb41df0bbd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:23.474300 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ac0b4-07ec-405f-888e-bb41df0bbd91 admin admin] PATH_INFO: `/v3/users/11f54ae8d51c49489891bf4fa6841d63` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:23.475753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ac0b4-07ec-405f-888e-bb41df0bbd91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11f54ae8d51c49489891bf4fa6841d63)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:23.479305 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ac0b4-07ec-405f-888e-bb41df0bbd91 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:23.489213 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:23.537427 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1055/2135] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:23 2022] POST /identity/v3/projects => generated 377 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:23.538921 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b56ba-a2e5-4378-9548-ea5759dc76ce admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:23.612309 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b56ba-a2e5-4378-9548-ea5759dc76ce admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:23.620273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a5012-6601-410b-8050-38e44a116323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:23.621108 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a5012-6601-410b-8050-38e44a116323 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:23.621521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a5012-6601-410b-8050-38e44a116323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:23.621801 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a5012-6601-410b-8050-38e44a116323 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:23.622629 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a5012-6601-410b-8050-38e44a116323 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:39:23.622942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a5012-6601-410b-8050-38e44a116323 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:23.624743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a5012-6601-410b-8050-38e44a116323 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:23.655069 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1056/2136] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:39:23.655069 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a5012-6601-410b-8050-38e44a116323 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:23.656911 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4c7a5012-6601-410b-8050-38e44a116323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:23.675206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a5012-6601-410b-8050-38e44a116323 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:23.680425 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0a3c6-1227-405d-904e-4ac20fdc4e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:23.693519 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0a3c6-1227-405d-904e-4ac20fdc4e4c admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:23.693871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0a3c6-1227-405d-904e-4ac20fdc4e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:23.694118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0a3c6-1227-405d-904e-4ac20fdc4e4c admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:23.694867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0a3c6-1227-405d-904e-4ac20fdc4e4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:23.696875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0a3c6-1227-405d-904e-4ac20fdc4e4c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:23.704173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e41ac0b4-07ec-405f-888e-bb41df0bbd91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11f54ae8d51c49489891bf4fa6841d63'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:23.704963 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e41ac0b4-07ec-405f-888e-bb41df0bbd91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11f54ae8d51c49489891bf4fa6841d63'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:23.713769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e41ac0b4-07ec-405f-888e-bb41df0bbd91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11f54ae8d51c49489891bf4fa6841d63'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:23.714643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e41ac0b4-07ec-405f-888e-bb41df0bbd91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11f54ae8d51c49489891bf4fa6841d63'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:23.833731 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1057/2137] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:23 2022] POST /identity/v3/users => generated 295 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:23.834952 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0a3c6-1227-405d-904e-4ac20fdc4e4c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:23.836986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bfc0a3c6-1227-405d-904e-4ac20fdc4e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:23.837195 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e41ac0b4-07ec-405f-888e-bb41df0bbd91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11f54ae8d51c49489891bf4fa6841d63'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:23.839979 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:23.875730 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0a3c6-1227-405d-904e-4ac20fdc4e4c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:23.882933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1081/2138] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:23 2022] DELETE /identity/v3/users/11f54ae8d51c49489891bf4fa6841d63 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:23.884815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ac0b4-07ec-405f-888e-bb41df0bbd91 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:23.889406 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e41ac0b4-07ec-405f-888e-bb41df0bbd91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:23.891216 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ab651-5d76-4e3e-a7b3-4e997db97a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:23.891904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ab651-5d76-4e3e-a7b3-4e997db97a4e admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:23.893767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ab651-5d76-4e3e-a7b3-4e997db97a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:23.893767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ab651-5d76-4e3e-a7b3-4e997db97a4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:23.893767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ab651-5d76-4e3e-a7b3-4e997db97a4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:23.895272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ab651-5d76-4e3e-a7b3-4e997db97a4e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:24.009120 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1058/2139] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:23 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:24.011300 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d27202-8a42-40a1-823f-de05830c09a4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:24.011813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d27202-8a42-40a1-823f-de05830c09a4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:24.012282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d27202-8a42-40a1-823f-de05830c09a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:24.067769 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f2d27202-8a42-40a1-823f-de05830c09a4 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:24.070593 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f2d27202-8a42-40a1-823f-de05830c09a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:24.117248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ac0b4-07ec-405f-888e-bb41df0bbd91 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:24.208138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1059/2140] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:24.222125 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d27202-8a42-40a1-823f-de05830c09a4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:24.261608 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d27202-8a42-40a1-823f-de05830c09a4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:24.274350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed55f2-7a8a-459d-93d0-152069d4ab03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:24.281339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed55f2-7a8a-459d-93d0-152069d4ab03 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:24.281339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed55f2-7a8a-459d-93d0-152069d4ab03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:24.281339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed55f2-7a8a-459d-93d0-152069d4ab03 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:24.281339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed55f2-7a8a-459d-93d0-152069d4ab03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:24.284926 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f089951-9322-48e6-b962-25f669496a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:24.285739 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f089951-9322-48e6-b962-25f669496a2e admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:24.286029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f089951-9322-48e6-b962-25f669496a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:24.286302 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f089951-9322-48e6-b962-25f669496a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:24.287556 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed55f2-7a8a-459d-93d0-152069d4ab03 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:24.287680 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f089951-9322-48e6-b962-25f669496a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:24.289292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f089951-9322-48e6-b962-25f669496a2e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:24.317678 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1082/2141] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:24.319682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f089951-9322-48e6-b962-25f669496a2e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:24.322305 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2f089951-9322-48e6-b962-25f669496a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:24.383769 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1060/2142] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:24 2022] POST /identity/v3/users => generated 292 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:24.383769 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed55f2-7a8a-459d-93d0-152069d4ab03 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:24.389306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f089951-9322-48e6-b962-25f669496a2e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:24.391102 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed55f2-7a8a-459d-93d0-152069d4ab03 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:24.393078 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af039f-6124-4258-b191-f1916873192e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:24.393740 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af039f-6124-4258-b191-f1916873192e service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:24.394020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af039f-6124-4258-b191-f1916873192e service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:24.394290 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af039f-6124-4258-b191-f1916873192e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:24.395963 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c5cd0-a374-40da-ae05-b976fdb799c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:24.396802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c5cd0-a374-40da-ae05-b976fdb799c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:24.397238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c5cd0-a374-40da-ae05-b976fdb799c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:24.397358 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c5cd0-a374-40da-ae05-b976fdb799c8 admin admin] PATH_INFO: `/v3/projects/8a9fddf8a3c24c2c914dc8640fcea2a3/users/6dacc2cb87094a1fb8f4e58d412ce855/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:24.406506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af039f-6124-4258-b191-f1916873192e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:24.408938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af039f-6124-4258-b191-f1916873192e service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:24.417503 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c5cd0-a374-40da-ae05-b976fdb799c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9fddf8a3c24c2c914dc8640fcea2a3, user_id=6dacc2cb87094a1fb8f4e58d412ce855, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:24.420835 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c5cd0-a374-40da-ae05-b976fdb799c8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:24.435512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1083/2143] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:24.440872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af039f-6124-4258-b191-f1916873192e service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:24.456845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af039f-6124-4258-b191-f1916873192e service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:24.462557 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54c617-559b-48bb-afd9-9874f694add5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:24.463716 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54c617-559b-48bb-afd9-9874f694add5 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:24.464075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54c617-559b-48bb-afd9-9874f694add5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:24.464439 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54c617-559b-48bb-afd9-9874f694add5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:24.465304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54c617-559b-48bb-afd9-9874f694add5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:24.467818 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54c617-559b-48bb-afd9-9874f694add5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:24.507940 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1061/2144] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:24 2022] PUT /identity/v3/projects/8a9fddf8a3c24c2c914dc8640fcea2a3/users/6dacc2cb87094a1fb8f4e58d412ce855/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:24.512677 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c5cd0-a374-40da-ae05-b976fdb799c8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:24.532987 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1084/2145] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:24.540948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54c617-559b-48bb-afd9-9874f694add5 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:24.569548 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54c617-559b-48bb-afd9-9874f694add5 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:24.717301 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c5cd0-a374-40da-ae05-b976fdb799c8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:24.725714 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b13fe-dee9-4dcf-9fca-0180b41a6e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:24.726764 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b13fe-dee9-4dcf-9fca-0180b41a6e34 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:24.727190 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b13fe-dee9-4dcf-9fca-0180b41a6e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:24.727314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b13fe-dee9-4dcf-9fca-0180b41a6e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:24.728048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b13fe-dee9-4dcf-9fca-0180b41a6e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:24.729980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:24.731439 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3451c3-7608-423b-8bac-d2e3686dd3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:24.732796 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3451c3-7608-423b-8bac-d2e3686dd3fa admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:24.733168 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3451c3-7608-423b-8bac-d2e3686dd3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:24.733453 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3451c3-7608-423b-8bac-d2e3686dd3fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:24.736451 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3451c3-7608-423b-8bac-d2e3686dd3fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:24.738475 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3451c3-7608-423b-8bac-d2e3686dd3fa admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:24.740759 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b13fe-dee9-4dcf-9fca-0180b41a6e34 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:24.755779 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1085/2146] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:24.758220 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1062/2147] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:24.772815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b13fe-dee9-4dcf-9fca-0180b41a6e34 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:24.774041 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3451c3-7608-423b-8bac-d2e3686dd3fa admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:24.805038 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3451c3-7608-423b-8bac-d2e3686dd3fa admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:24.811240 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b13fe-dee9-4dcf-9fca-0180b41a6e34 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:24.825857 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a06165-efc7-4558-82b8-db3be840b0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:24.826821 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a06165-efc7-4558-82b8-db3be840b0c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:24.827178 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a06165-efc7-4558-82b8-db3be840b0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:24.827491 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a06165-efc7-4558-82b8-db3be840b0c1 admin admin] PATH_INFO: `/v3/projects/8a9fddf8a3c24c2c914dc8640fcea2a3/users/6dacc2cb87094a1fb8f4e58d412ce855/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:24.830195 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fde9c-45de-4f89-b311-df54c3bfced1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:24.831710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fde9c-45de-4f89-b311-df54c3bfced1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:24.831968 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fde9c-45de-4f89-b311-df54c3bfced1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:24.832337 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fde9c-45de-4f89-b311-df54c3bfced1 admin admin] PATH_INFO: `/v3/projects/64afb373fb4349619b34e7cfe2323bb6/users/f7b7f37c007a4bb39634160de8845317/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:24.835954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a06165-efc7-4558-82b8-db3be840b0c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9fddf8a3c24c2c914dc8640fcea2a3, user_id=6dacc2cb87094a1fb8f4e58d412ce855, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:24.839422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a06165-efc7-4558-82b8-db3be840b0c1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:24.879316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1063/2148] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:24 2022] PUT /identity/v3/projects/8a9fddf8a3c24c2c914dc8640fcea2a3/users/6dacc2cb87094a1fb8f4e58d412ce855/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:24.899655 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fde9c-45de-4f89-b311-df54c3bfced1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64afb373fb4349619b34e7cfe2323bb6, user_id=f7b7f37c007a4bb39634160de8845317, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:24.901441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d9978-54ff-4d94-ba58-5a1385c414a5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:24.903058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d9978-54ff-4d94-ba58-5a1385c414a5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:24.903230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fde9c-45de-4f89-b311-df54c3bfced1 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:24.903394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d9978-54ff-4d94-ba58-5a1385c414a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:24.969464 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2b5d9978-54ff-4d94-ba58-5a1385c414a5 None None] MFA Rules not processed for user `6dacc2cb87094a1fb8f4e58d412ce855`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:24.971654 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2b5d9978-54ff-4d94-ba58-5a1385c414a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:24.996187 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1086/2149] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:24 2022] PUT /identity/v3/projects/64afb373fb4349619b34e7cfe2323bb6/users/f7b7f37c007a4bb39634160de8845317/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:25.010406 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8abe6-b32f-4228-8b9b-57e7a7ad6f17 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:25.010613 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8abe6-b32f-4228-8b9b-57e7a7ad6f17 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:25.010913 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8abe6-b32f-4228-8b9b-57e7a7ad6f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:25.077764 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-6ee8abe6-b32f-4228-8b9b-57e7a7ad6f17 None None] MFA Rules not processed for user `f7b7f37c007a4bb39634160de8845317`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:25.083048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6ee8abe6-b32f-4228-8b9b-57e7a7ad6f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:25.125397 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1064/2150] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:25.139281 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d9978-54ff-4d94-ba58-5a1385c414a5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:25.147267 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2b5d9978-54ff-4d94-ba58-5a1385c414a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:25.236444 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1087/2151] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:25.248136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8abe6-b32f-4228-8b9b-57e7a7ad6f17 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:25.249541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6ee8abe6-b32f-4228-8b9b-57e7a7ad6f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:25.272208 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d9978-54ff-4d94-ba58-5a1385c414a5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:25.335954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8abe6-b32f-4228-8b9b-57e7a7ad6f17 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:25.340364 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b1ee3-80fa-4145-b24c-d3aa49b02478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:25.341067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b1ee3-80fa-4145-b24c-d3aa49b02478 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:25.341356 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b1ee3-80fa-4145-b24c-d3aa49b02478 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:25.341635 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b1ee3-80fa-4145-b24c-d3aa49b02478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:25.342054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943e493-71d7-42fe-acab-0837269e9325 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:25.342706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943e493-71d7-42fe-acab-0837269e9325 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:25.343017 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943e493-71d7-42fe-acab-0837269e9325 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:25.343833 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7e2b1ee3-80fa-4145-b24c-d3aa49b02478 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:25.354778 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943e493-71d7-42fe-acab-0837269e9325 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:25.377040 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943e493-71d7-42fe-acab-0837269e9325 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:25.379361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943e493-71d7-42fe-acab-0837269e9325 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:25.394473 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1088/2152] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:39:25.396673 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943e493-71d7-42fe-acab-0837269e9325 service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:25.399362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5943e493-71d7-42fe-acab-0837269e9325 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:25.460657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b1ee3-80fa-4145-b24c-d3aa49b02478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:25.462533 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b1ee3-80fa-4145-b24c-d3aa49b02478 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:25.495647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1065/2153] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:25.515307 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943e493-71d7-42fe-acab-0837269e9325 service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:25.521871 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75ebb9-9fc9-4476-a747-cb19a2be6caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:25.522977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75ebb9-9fc9-4476-a747-cb19a2be6caa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:25.523095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75ebb9-9fc9-4476-a747-cb19a2be6caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:25.523330 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75ebb9-9fc9-4476-a747-cb19a2be6caa admin admin] PATH_INFO: `/v3/projects/d2432f00db7f413b88df47d89276cb05` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:25.527092 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75ebb9-9fc9-4476-a747-cb19a2be6caa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2432f00db7f413b88df47d89276cb05)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:25.529223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75ebb9-9fc9-4476-a747-cb19a2be6caa admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:25.682606 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5d75ebb9-9fc9-4476-a747-cb19a2be6caa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2432f00db7f413b88df47d89276cb05', 'request_id': 'req-5d75ebb9-9fc9-4476-a747-cb19a2be6caa'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:25.694120 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5d75ebb9-9fc9-4476-a747-cb19a2be6caa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2432f00db7f413b88df47d89276cb05', 'request_id': 'req-5d75ebb9-9fc9-4476-a747-cb19a2be6caa'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:25.826281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5d75ebb9-9fc9-4476-a747-cb19a2be6caa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2432f00db7f413b88df47d89276cb05', 'request_id': 'req-5d75ebb9-9fc9-4476-a747-cb19a2be6caa'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:25.847044 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5d75ebb9-9fc9-4476-a747-cb19a2be6caa admin admin] The token cache is being invalidate because project d2432f00db7f413b88df47d89276cb05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:25.847472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5d75ebb9-9fc9-4476-a747-cb19a2be6caa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:25.850318 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5d75ebb9-9fc9-4476-a747-cb19a2be6caa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:25.856229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1089/2154] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:25 2022] DELETE /identity/v3/projects/d2432f00db7f413b88df47d89276cb05 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:25.863202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b1ee3-80fa-4145-b24c-d3aa49b02478 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:25.893610 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b1ee3-80fa-4145-b24c-d3aa49b02478 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:26.009211 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85b21d-7c95-49ee-9c05-ad4119255b80 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:26.009211 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85b21d-7c95-49ee-9c05-ad4119255b80 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:26.009211 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85b21d-7c95-49ee-9c05-ad4119255b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:26.043388 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a231c-53de-42d6-af07-c370184641a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:26.047655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a231c-53de-42d6-af07-c370184641a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:26.047655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a231c-53de-42d6-af07-c370184641a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:26.047655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a231c-53de-42d6-af07-c370184641a8 admin admin] PATH_INFO: `/v3/users/8f1ec8b7b13c4662b8581944115090b8` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:26.047655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a231c-53de-42d6-af07-c370184641a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f1ec8b7b13c4662b8581944115090b8)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:26.049831 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a231c-53de-42d6-af07-c370184641a8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:26.057855 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9e85b21d-7c95-49ee-9c05-ad4119255b80 None None] MFA Rules not processed for user `4e5893f0909446ff82b5a72584ecae37`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:26.059349 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9e85b21d-7c95-49ee-9c05-ad4119255b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:26.129082 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:26.187003 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1090/2155] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:26.205689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85b21d-7c95-49ee-9c05-ad4119255b80 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:26.214959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9e85b21d-7c95-49ee-9c05-ad4119255b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:26.285328 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:26.315040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-670a231c-53de-42d6-af07-c370184641a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f1ec8b7b13c4662b8581944115090b8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:26.316414 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-670a231c-53de-42d6-af07-c370184641a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f1ec8b7b13c4662b8581944115090b8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:26.332981 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-670a231c-53de-42d6-af07-c370184641a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f1ec8b7b13c4662b8581944115090b8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:26.335105 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-670a231c-53de-42d6-af07-c370184641a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f1ec8b7b13c4662b8581944115090b8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:26.357367 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85b21d-7c95-49ee-9c05-ad4119255b80 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:26.357603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-670a231c-53de-42d6-af07-c370184641a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f1ec8b7b13c4662b8581944115090b8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:26.359276 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992b04a-3161-48dc-a117-9692469905fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:26.360048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992b04a-3161-48dc-a117-9692469905fd service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:26.360429 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992b04a-3161-48dc-a117-9692469905fd service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:26.360710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992b04a-3161-48dc-a117-9692469905fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:26.362929 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6992b04a-3161-48dc-a117-9692469905fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:26.401037 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992b04a-3161-48dc-a117-9692469905fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:26.403070 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992b04a-3161-48dc-a117-9692469905fd service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:26.426508 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1091/2156] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:26.499547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1066/2157] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:25 2022] DELETE /identity/v3/users/8f1ec8b7b13c4662b8581944115090b8 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:27.001053 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992b04a-3161-48dc-a117-9692469905fd service nova] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:27.002669 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6992b04a-3161-48dc-a117-9692469905fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:27.024337 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992b04a-3161-48dc-a117-9692469905fd service nova] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:27.082031 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d75fd-098e-41b4-a8fe-243abaf1f18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:27.082704 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d75fd-098e-41b4-a8fe-243abaf1f18d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:27.082984 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d75fd-098e-41b4-a8fe-243abaf1f18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:27.083397 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d75fd-098e-41b4-a8fe-243abaf1f18d admin admin] PATH_INFO: `/v3/projects/f3f7f33ad81045038af1adf000797229` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:27.085357 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d75fd-098e-41b4-a8fe-243abaf1f18d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3f7f33ad81045038af1adf000797229)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:27.087335 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d75fd-098e-41b4-a8fe-243abaf1f18d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:27.367534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-689d75fd-098e-41b4-a8fe-243abaf1f18d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3f7f33ad81045038af1adf000797229', 'request_id': 'req-689d75fd-098e-41b4-a8fe-243abaf1f18d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:27.376198 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-689d75fd-098e-41b4-a8fe-243abaf1f18d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3f7f33ad81045038af1adf000797229', 'request_id': 'req-689d75fd-098e-41b4-a8fe-243abaf1f18d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:27.510818 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-689d75fd-098e-41b4-a8fe-243abaf1f18d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3f7f33ad81045038af1adf000797229', 'request_id': 'req-689d75fd-098e-41b4-a8fe-243abaf1f18d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:27.521751 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-689d75fd-098e-41b4-a8fe-243abaf1f18d admin admin] The token cache is being invalidate because project f3f7f33ad81045038af1adf000797229 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:27.522032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-689d75fd-098e-41b4-a8fe-243abaf1f18d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:27.523350 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-689d75fd-098e-41b4-a8fe-243abaf1f18d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:27.525523 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1092/2158] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:26 2022] DELETE /identity/v3/projects/f3f7f33ad81045038af1adf000797229 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:27.536339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17c2d0-0930-40ef-ac13-6cef4caf84ed None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:27.537375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17c2d0-0930-40ef-ac13-6cef4caf84ed None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:27.538436 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17c2d0-0930-40ef-ac13-6cef4caf84ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:27.558847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3a17c2d0-0930-40ef-ac13-6cef4caf84ed None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:27.560066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3a17c2d0-0930-40ef-ac13-6cef4caf84ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:27.629389 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1067/2159] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:27.643143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f3c84-9174-41bc-a85b-4b8509e3c77a None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:27.643469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f3c84-9174-41bc-a85b-4b8509e3c77a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:27.643729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f3c84-9174-41bc-a85b-4b8509e3c77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:27.674570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-010f3c84-9174-41bc-a85b-4b8509e3c77a None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:27.676095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-010f3c84-9174-41bc-a85b-4b8509e3c77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:27.686209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1093/2160] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:27.690549 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17c2d0-0930-40ef-ac13-6cef4caf84ed None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:27.701304 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17c2d0-0930-40ef-ac13-6cef4caf84ed None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:27.705915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b978c63-a5a2-457f-aa14-bed78da7f27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:27.706752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b978c63-a5a2-457f-aa14-bed78da7f27c admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:27.707038 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b978c63-a5a2-457f-aa14-bed78da7f27c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:27.707424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b978c63-a5a2-457f-aa14-bed78da7f27c admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:27.708088 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b978c63-a5a2-457f-aa14-bed78da7f27c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:39:27.708391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b978c63-a5a2-457f-aa14-bed78da7f27c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:27.710068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b978c63-a5a2-457f-aa14-bed78da7f27c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:27.722093 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1068/2161] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:39:27.726837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f3c84-9174-41bc-a85b-4b8509e3c77a None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:27.736669 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f3c84-9174-41bc-a85b-4b8509e3c77a None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:27.742041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5869c3c-8ffe-45d7-9c41-05d1338aa899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:27.742725 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5869c3c-8ffe-45d7-9c41-05d1338aa899 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:27.743007 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5869c3c-8ffe-45d7-9c41-05d1338aa899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:27.743280 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5869c3c-8ffe-45d7-9c41-05d1338aa899 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:27.744062 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5869c3c-8ffe-45d7-9c41-05d1338aa899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:27.746098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5869c3c-8ffe-45d7-9c41-05d1338aa899 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:27.839819 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1094/2162] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:27 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:27.842016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b978c63-a5a2-457f-aa14-bed78da7f27c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:27.852497 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b978c63-a5a2-457f-aa14-bed78da7f27c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:27.929423 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecd9f3-0b9a-4009-b75c-c91e4b82040d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:27.930116 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecd9f3-0b9a-4009-b75c-c91e4b82040d admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:27.930401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecd9f3-0b9a-4009-b75c-c91e4b82040d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:27.930676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecd9f3-0b9a-4009-b75c-c91e4b82040d admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:27.931382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aecd9f3-0b9a-4009-b75c-c91e4b82040d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:27.933498 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aecd9f3-0b9a-4009-b75c-c91e4b82040d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:28.017061 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1069/2163] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:27 2022] POST /identity/v3/users => generated 287 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:28.021302 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5869c3c-8ffe-45d7-9c41-05d1338aa899 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:28.032723 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5869c3c-8ffe-45d7-9c41-05d1338aa899 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:28.039327 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d681f4-0b6f-43ca-be29-b8a6ee943dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:28.040373 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d681f4-0b6f-43ca-be29-b8a6ee943dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:28.040915 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d681f4-0b6f-43ca-be29-b8a6ee943dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:28.041449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d681f4-0b6f-43ca-be29-b8a6ee943dc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:28.042438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d681f4-0b6f-43ca-be29-b8a6ee943dc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:28.044203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d681f4-0b6f-43ca-be29-b8a6ee943dc6 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:28.056368 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1095/2164] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:39:28.060541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecd9f3-0b9a-4009-b75c-c91e4b82040d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:28.073100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecd9f3-0b9a-4009-b75c-c91e4b82040d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:28.080769 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800af033-1160-4ca4-937c-bf3848fac849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:28.081593 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800af033-1160-4ca4-937c-bf3848fac849 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:28.081950 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800af033-1160-4ca4-937c-bf3848fac849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:28.082238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800af033-1160-4ca4-937c-bf3848fac849 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:28.083079 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800af033-1160-4ca4-937c-bf3848fac849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:28.084854 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800af033-1160-4ca4-937c-bf3848fac849 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:28.096139 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1070/2165] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:28.100961 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d681f4-0b6f-43ca-be29-b8a6ee943dc6 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:28.115952 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d681f4-0b6f-43ca-be29-b8a6ee943dc6 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:28.121314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0e560-eea6-4e2b-909d-542ca810ca9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:28.122339 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0e560-eea6-4e2b-909d-542ca810ca9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:28.122648 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0e560-eea6-4e2b-909d-542ca810ca9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:28.122921 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0e560-eea6-4e2b-909d-542ca810ca9a admin admin] PATH_INFO: `/v3/projects/6e75ec518a3946dcb348036134e7bec4/users/77328d8f2f9f47d3a01fb482b613c060/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:28.140835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0e560-eea6-4e2b-909d-542ca810ca9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e75ec518a3946dcb348036134e7bec4, user_id=77328d8f2f9f47d3a01fb482b613c060, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:28.144017 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0e560-eea6-4e2b-909d-542ca810ca9a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:28.146350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:28.240372 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1096/2166] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:28 2022] PUT /identity/v3/projects/6e75ec518a3946dcb348036134e7bec4/users/77328d8f2f9f47d3a01fb482b613c060/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:28.242701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dfeba-c1c3-445c-bb39-1b0d4e20bd28 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:28.243252 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dfeba-c1c3-445c-bb39-1b0d4e20bd28 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:28.243703 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dfeba-c1c3-445c-bb39-1b0d4e20bd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:28.281316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9a2dfeba-c1c3-445c-bb39-1b0d4e20bd28 None None] MFA Rules not processed for user `77328d8f2f9f47d3a01fb482b613c060`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:28.282712 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9a2dfeba-c1c3-445c-bb39-1b0d4e20bd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:28.366771 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1071/2167] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:28.374371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0e560-eea6-4e2b-909d-542ca810ca9a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:28.376378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-33f0e560-eea6-4e2b-909d-542ca810ca9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:28.444964 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0e560-eea6-4e2b-909d-542ca810ca9a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:28.449477 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e602f-2dda-4817-99c3-0d07156b851e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:28.450201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e602f-2dda-4817-99c3-0d07156b851e service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:28.450598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e602f-2dda-4817-99c3-0d07156b851e service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:28.450805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e602f-2dda-4817-99c3-0d07156b851e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:28.461619 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e602f-2dda-4817-99c3-0d07156b851e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:28.463039 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e602f-2dda-4817-99c3-0d07156b851e service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:28.475213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1097/2168] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:28.502151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2dfeba-c1c3-445c-bb39-1b0d4e20bd28 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:28.600841 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2dfeba-c1c3-445c-bb39-1b0d4e20bd28 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:28.615701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca22a82-dae2-4372-be1d-c957a79e0ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:28.616556 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca22a82-dae2-4372-be1d-c957a79e0ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:28.616838 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca22a82-dae2-4372-be1d-c957a79e0ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:28.620652 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca22a82-dae2-4372-be1d-c957a79e0ddf admin admin] PATH_INFO: `/v3/users/77328d8f2f9f47d3a01fb482b613c060` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:28.624405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca22a82-dae2-4372-be1d-c957a79e0ddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=77328d8f2f9f47d3a01fb482b613c060)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:28.628812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca22a82-dae2-4372-be1d-c957a79e0ddf admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:28.856079 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:28.875496 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2ca22a82-dae2-4372-be1d-c957a79e0ddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77328d8f2f9f47d3a01fb482b613c060'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:28.876799 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2ca22a82-dae2-4372-be1d-c957a79e0ddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77328d8f2f9f47d3a01fb482b613c060'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:28.889422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2ca22a82-dae2-4372-be1d-c957a79e0ddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77328d8f2f9f47d3a01fb482b613c060'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:28.890514 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2ca22a82-dae2-4372-be1d-c957a79e0ddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77328d8f2f9f47d3a01fb482b613c060'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:28.909026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2ca22a82-dae2-4372-be1d-c957a79e0ddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77328d8f2f9f47d3a01fb482b613c060'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:28.975021 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e602f-2dda-4817-99c3-0d07156b851e service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:28.976910 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1a7e602f-2dda-4817-99c3-0d07156b851e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:28.988668 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e602f-2dda-4817-99c3-0d07156b851e service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:28.993197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6395f-7dc2-46d7-b0a1-2700b13c5a8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:28.994206 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6395f-7dc2-46d7-b0a1-2700b13c5a8e service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:28.994584 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6395f-7dc2-46d7-b0a1-2700b13c5a8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:28.995078 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6395f-7dc2-46d7-b0a1-2700b13c5a8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:28.997318 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f2e6395f-7dc2-46d7-b0a1-2700b13c5a8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:29.003765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1072/2169] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:28 2022] DELETE /identity/v3/users/77328d8f2f9f47d3a01fb482b613c060 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:29.009343 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242134c0-0e6f-418e-827c-acceea6e9fe4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:29.010459 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242134c0-0e6f-418e-827c-acceea6e9fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:29.010776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242134c0-0e6f-418e-827c-acceea6e9fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:29.049172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-242134c0-0e6f-418e-827c-acceea6e9fe4 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:29.051634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-242134c0-0e6f-418e-827c-acceea6e9fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:29.119100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6395f-7dc2-46d7-b0a1-2700b13c5a8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:29.122298 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6395f-7dc2-46d7-b0a1-2700b13c5a8e service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:29.136770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1098/2170] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:39:29.180769 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1073/2171] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:29.192092 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6395f-7dc2-46d7-b0a1-2700b13c5a8e service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:29.195983 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f2e6395f-7dc2-46d7-b0a1-2700b13c5a8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:29.370658 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6395f-7dc2-46d7-b0a1-2700b13c5a8e service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:29.485341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a36ea-ba0b-46a7-8e8f-11fd3449fca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:29.486360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a36ea-ba0b-46a7-8e8f-11fd3449fca3 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:29.486959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a36ea-ba0b-46a7-8e8f-11fd3449fca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:29.487207 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a36ea-ba0b-46a7-8e8f-11fd3449fca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:29.503129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a36ea-ba0b-46a7-8e8f-11fd3449fca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:29.506033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a36ea-ba0b-46a7-8e8f-11fd3449fca3 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:29.524448 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1099/2172] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:29.694996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9595b1-4402-4e0f-bcf4-a0f98a8d6cbc None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:29.695708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9595b1-4402-4e0f-bcf4-a0f98a8d6cbc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:29.695708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9595b1-4402-4e0f-bcf4-a0f98a8d6cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:29.722958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4d9595b1-4402-4e0f-bcf4-a0f98a8d6cbc None None] MFA Rules not processed for user `4e5893f0909446ff82b5a72584ecae37`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:29.724906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4d9595b1-4402-4e0f-bcf4-a0f98a8d6cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:29.794422 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1074/2173] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:30.242433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a36ea-ba0b-46a7-8e8f-11fd3449fca3 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:30.243924 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e99a36ea-ba0b-46a7-8e8f-11fd3449fca3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:30.289689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a36ea-ba0b-46a7-8e8f-11fd3449fca3 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:30.300460 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843f5bd-8fcf-4674-b8d5-fd5433920baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:30.303512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843f5bd-8fcf-4674-b8d5-fd5433920baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:30.303804 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843f5bd-8fcf-4674-b8d5-fd5433920baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:30.304085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843f5bd-8fcf-4674-b8d5-fd5433920baf admin admin] PATH_INFO: `/v3/projects/6e75ec518a3946dcb348036134e7bec4` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:30.313643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843f5bd-8fcf-4674-b8d5-fd5433920baf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e75ec518a3946dcb348036134e7bec4)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:30.313643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843f5bd-8fcf-4674-b8d5-fd5433920baf admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:30.353157 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:30.425798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:30.470209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a843f5bd-8fcf-4674-b8d5-fd5433920baf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e75ec518a3946dcb348036134e7bec4', 'request_id': 'req-a843f5bd-8fcf-4674-b8d5-fd5433920baf'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:30.480794 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a843f5bd-8fcf-4674-b8d5-fd5433920baf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e75ec518a3946dcb348036134e7bec4', 'request_id': 'req-a843f5bd-8fcf-4674-b8d5-fd5433920baf'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:30.556701 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a843f5bd-8fcf-4674-b8d5-fd5433920baf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e75ec518a3946dcb348036134e7bec4', 'request_id': 'req-a843f5bd-8fcf-4674-b8d5-fd5433920baf'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:30.571644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a843f5bd-8fcf-4674-b8d5-fd5433920baf admin admin] The token cache is being invalidate because project 6e75ec518a3946dcb348036134e7bec4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:30.571875 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a843f5bd-8fcf-4674-b8d5-fd5433920baf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:30.572731 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a843f5bd-8fcf-4674-b8d5-fd5433920baf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:30.579098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1100/2174] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:30 2022] DELETE /identity/v3/projects/6e75ec518a3946dcb348036134e7bec4 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:39:30.580368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51f3c8-88cb-46aa-9897-2b6655f74685 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:30.580664 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51f3c8-88cb-46aa-9897-2b6655f74685 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:30.580949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51f3c8-88cb-46aa-9897-2b6655f74685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:30.602708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2c51f3c8-88cb-46aa-9897-2b6655f74685 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:30.604693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2c51f3c8-88cb-46aa-9897-2b6655f74685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:30.672132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1075/2175] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:30.681121 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8e2be-dddc-4bb3-a48f-27b213b58d7c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:30.681669 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8e2be-dddc-4bb3-a48f-27b213b58d7c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:30.681669 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8e2be-dddc-4bb3-a48f-27b213b58d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:30.711526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d9d8e2be-dddc-4bb3-a48f-27b213b58d7c None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:30.712905 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d9d8e2be-dddc-4bb3-a48f-27b213b58d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:30.721176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1101/2176] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:30.725727 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51f3c8-88cb-46aa-9897-2b6655f74685 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:30.739547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51f3c8-88cb-46aa-9897-2b6655f74685 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:30.747376 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d555ca-c715-4d17-85c0-387f4f8c7d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:30.748844 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d555ca-c715-4d17-85c0-387f4f8c7d1b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:30.749329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d555ca-c715-4d17-85c0-387f4f8c7d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:30.749756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d555ca-c715-4d17-85c0-387f4f8c7d1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:30.750644 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d555ca-c715-4d17-85c0-387f4f8c7d1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:39:30.751191 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d555ca-c715-4d17-85c0-387f4f8c7d1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:30.753249 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d555ca-c715-4d17-85c0-387f4f8c7d1b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:30.784743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1076/2177] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:39:30.788689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8e2be-dddc-4bb3-a48f-27b213b58d7c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:30.803365 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8e2be-dddc-4bb3-a48f-27b213b58d7c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:30.808916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eec99b-51c6-4877-b2c5-1250f1cb448f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:30.809860 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eec99b-51c6-4877-b2c5-1250f1cb448f admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:30.810232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eec99b-51c6-4877-b2c5-1250f1cb448f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:30.810632 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eec99b-51c6-4877-b2c5-1250f1cb448f admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:30.811567 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eec99b-51c6-4877-b2c5-1250f1cb448f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:30.814885 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eec99b-51c6-4877-b2c5-1250f1cb448f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:30.919119 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1102/2178] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:30 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:30.922599 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:30.924178 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d555ca-c715-4d17-85c0-387f4f8c7d1b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:30.935892 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d555ca-c715-4d17-85c0-387f4f8c7d1b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:31.008715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd0c48-813e-4b7d-b76f-66ae37b9331c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:31.009434 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd0c48-813e-4b7d-b76f-66ae37b9331c admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:31.009715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd0c48-813e-4b7d-b76f-66ae37b9331c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:31.010319 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd0c48-813e-4b7d-b76f-66ae37b9331c admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:31.010694 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd0c48-813e-4b7d-b76f-66ae37b9331c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:31.012782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd0c48-813e-4b7d-b76f-66ae37b9331c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:31.135397 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1077/2179] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:30 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:31.139642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eec99b-51c6-4877-b2c5-1250f1cb448f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:31.153074 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eec99b-51c6-4877-b2c5-1250f1cb448f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:31.169639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18fd65-4e1f-47c8-94b7-ecefc3b15760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:31.170343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18fd65-4e1f-47c8-94b7-ecefc3b15760 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:31.170604 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18fd65-4e1f-47c8-94b7-ecefc3b15760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:31.170879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18fd65-4e1f-47c8-94b7-ecefc3b15760 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:31.178398 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18fd65-4e1f-47c8-94b7-ecefc3b15760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:31.178398 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18fd65-4e1f-47c8-94b7-ecefc3b15760 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:31.188902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1103/2180] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:31.193332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd0c48-813e-4b7d-b76f-66ae37b9331c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:31.204068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd0c48-813e-4b7d-b76f-66ae37b9331c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:31.210991 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528488fa-a32f-49bd-903c-5bbdbda4fb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:31.210991 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528488fa-a32f-49bd-903c-5bbdbda4fb72 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:31.213206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528488fa-a32f-49bd-903c-5bbdbda4fb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:31.213206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528488fa-a32f-49bd-903c-5bbdbda4fb72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:31.213206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528488fa-a32f-49bd-903c-5bbdbda4fb72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:31.213865 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528488fa-a32f-49bd-903c-5bbdbda4fb72 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:31.224736 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1078/2181] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:31.229951 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18fd65-4e1f-47c8-94b7-ecefc3b15760 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:31.244456 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18fd65-4e1f-47c8-94b7-ecefc3b15760 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:31.250580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9af3cd-2ea1-417b-ad5f-145584ed4204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:31.251673 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9af3cd-2ea1-417b-ad5f-145584ed4204 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:31.251961 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9af3cd-2ea1-417b-ad5f-145584ed4204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:31.252267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9af3cd-2ea1-417b-ad5f-145584ed4204 admin admin] PATH_INFO: `/v3/projects/996acbb0d97c48a79cda471707947d85/users/5a91233b22f04eec99328e15e6e8c5ef/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:31.268870 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9af3cd-2ea1-417b-ad5f-145584ed4204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=996acbb0d97c48a79cda471707947d85, user_id=5a91233b22f04eec99328e15e6e8c5ef, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:31.271600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9af3cd-2ea1-417b-ad5f-145584ed4204 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:31.402769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1104/2182] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:31 2022] PUT /identity/v3/projects/996acbb0d97c48a79cda471707947d85/users/5a91233b22f04eec99328e15e6e8c5ef/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:31.414288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d39d09-0563-4592-be16-22e0490a7f5b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:31.414624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d39d09-0563-4592-be16-22e0490a7f5b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:31.414867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d39d09-0563-4592-be16-22e0490a7f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:31.442724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-16d39d09-0563-4592-be16-22e0490a7f5b None None] MFA Rules not processed for user `5a91233b22f04eec99328e15e6e8c5ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:31.444499 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-16d39d09-0563-4592-be16-22e0490a7f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:31.517394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1079/2183] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:31.531710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9af3cd-2ea1-417b-ad5f-145584ed4204 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:31.533961 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cf9af3cd-2ea1-417b-ad5f-145584ed4204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:31.592390 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9af3cd-2ea1-417b-ad5f-145584ed4204 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:31.596153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab085db-caf2-41b2-a09c-67659c74f1dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:31.596797 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab085db-caf2-41b2-a09c-67659c74f1dc service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:31.597110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab085db-caf2-41b2-a09c-67659c74f1dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:31.597412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab085db-caf2-41b2-a09c-67659c74f1dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:31.606636 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab085db-caf2-41b2-a09c-67659c74f1dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:31.608089 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab085db-caf2-41b2-a09c-67659c74f1dc service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:31.618828 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1105/2184] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:39:31.785930 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:32.212643 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d39d09-0563-4592-be16-22e0490a7f5b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:32.214331 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-16d39d09-0563-4592-be16-22e0490a7f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:32.226484 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d39d09-0563-4592-be16-22e0490a7f5b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:32.230024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5ca2a-97ee-4edf-8198-5aefa96be8a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:32.230664 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5ca2a-97ee-4edf-8198-5aefa96be8a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:32.230945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5ca2a-97ee-4edf-8198-5aefa96be8a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:32.231219 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5ca2a-97ee-4edf-8198-5aefa96be8a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:32.232927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-52d5ca2a-97ee-4edf-8198-5aefa96be8a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:32.303139 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5ca2a-97ee-4edf-8198-5aefa96be8a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:32.304605 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5ca2a-97ee-4edf-8198-5aefa96be8a1 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:32.314835 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1080/2185] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:32.482557 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:32.535649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:33.002504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:33.031682 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:33.322356 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=78,keystone:INSERT=5,keystone:DELETE=17 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:33.362158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:33.511357 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:33.539934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:33.636690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:33.830954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:33.888097 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:33.973908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:34.048145 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:34.076716 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:34.157114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:34.174398 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:34.322472 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:34.414368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:34.482077 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:36.873814 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab085db-caf2-41b2-a09c-67659c74f1dc service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:36.928711 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:36.957639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab085db-caf2-41b2-a09c-67659c74f1dc service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:36.969700 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6b6db-611a-485b-b924-32e034162b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:36.970600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6b6db-611a-485b-b924-32e034162b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:36.970880 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6b6db-611a-485b-b924-32e034162b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:36.971150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6b6db-611a-485b-b924-32e034162b04 admin admin] PATH_INFO: `/v3/users/5a91233b22f04eec99328e15e6e8c5ef` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:36.973841 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6b6db-611a-485b-b924-32e034162b04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a91233b22f04eec99328e15e6e8c5ef)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:36.976094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6b6db-611a-485b-b924-32e034162b04 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:37.289880 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d2f6b6db-611a-485b-b924-32e034162b04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a91233b22f04eec99328e15e6e8c5ef'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:37.290577 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d2f6b6db-611a-485b-b924-32e034162b04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a91233b22f04eec99328e15e6e8c5ef'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:37.302729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d2f6b6db-611a-485b-b924-32e034162b04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a91233b22f04eec99328e15e6e8c5ef'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:37.304013 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d2f6b6db-611a-485b-b924-32e034162b04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a91233b22f04eec99328e15e6e8c5ef'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:37.441455 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d2f6b6db-611a-485b-b924-32e034162b04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a91233b22f04eec99328e15e6e8c5ef'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:37.480611 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1106/2186] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:36 2022] DELETE /identity/v3/users/5a91233b22f04eec99328e15e6e8c5ef => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:37.486003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f83fef-2d47-49f0-80e6-e6aa08c1d50b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:37.490071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f83fef-2d47-49f0-80e6-e6aa08c1d50b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:37.490867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f83fef-2d47-49f0-80e6-e6aa08c1d50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:37.514198 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a4f83fef-2d47-49f0-80e6-e6aa08c1d50b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:37.515921 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a4f83fef-2d47-49f0-80e6-e6aa08c1d50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:37.596230 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1081/2187] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:37.606953 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6b6db-611a-485b-b924-32e034162b04 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:37.608691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d2f6b6db-611a-485b-b924-32e034162b04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:37.686395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6b6db-611a-485b-b924-32e034162b04 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:37.691156 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02cb012-49ef-425c-9a49-824b77a17bc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:37.691814 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cb012-49ef-425c-9a49-824b77a17bc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:37.692420 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cb012-49ef-425c-9a49-824b77a17bc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:37.692725 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cb012-49ef-425c-9a49-824b77a17bc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:37.707276 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02cb012-49ef-425c-9a49-824b77a17bc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:37.709623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02cb012-49ef-425c-9a49-824b77a17bc2 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:37.725101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1107/2188] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:38.221057 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f83fef-2d47-49f0-80e6-e6aa08c1d50b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:38.223136 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a4f83fef-2d47-49f0-80e6-e6aa08c1d50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:38.245452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f83fef-2d47-49f0-80e6-e6aa08c1d50b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:38.251395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c75d4-3d69-4751-b215-69b85aa7bfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:38.252313 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c75d4-3d69-4751-b215-69b85aa7bfda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:38.252593 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c75d4-3d69-4751-b215-69b85aa7bfda admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:38.252867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c75d4-3d69-4751-b215-69b85aa7bfda admin admin] PATH_INFO: `/v3/projects/996acbb0d97c48a79cda471707947d85` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:38.254111 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c75d4-3d69-4751-b215-69b85aa7bfda admin admin] RBAC: Authorizing `identity:delete_project(project_id=996acbb0d97c48a79cda471707947d85)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:38.256163 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c75d4-3d69-4751-b215-69b85aa7bfda admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:38.391393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e41c75d4-3d69-4751-b215-69b85aa7bfda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '996acbb0d97c48a79cda471707947d85', 'request_id': 'req-e41c75d4-3d69-4751-b215-69b85aa7bfda'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:38.402332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e41c75d4-3d69-4751-b215-69b85aa7bfda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '996acbb0d97c48a79cda471707947d85', 'request_id': 'req-e41c75d4-3d69-4751-b215-69b85aa7bfda'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:38.412541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e41c75d4-3d69-4751-b215-69b85aa7bfda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '996acbb0d97c48a79cda471707947d85', 'request_id': 'req-e41c75d4-3d69-4751-b215-69b85aa7bfda'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:38.531935 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e41c75d4-3d69-4751-b215-69b85aa7bfda admin admin] The token cache is being invalidate because project 996acbb0d97c48a79cda471707947d85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:38.531935 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e41c75d4-3d69-4751-b215-69b85aa7bfda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:38.531935 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e41c75d4-3d69-4751-b215-69b85aa7bfda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:38.531935 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1082/2189] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:38 2022] DELETE /identity/v3/projects/996acbb0d97c48a79cda471707947d85 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:39.483068 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02cb012-49ef-425c-9a49-824b77a17bc2 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:39.484849 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f02cb012-49ef-425c-9a49-824b77a17bc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:39.503791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02cb012-49ef-425c-9a49-824b77a17bc2 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:39.579461 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b3e16-dd6c-4c9f-a3bc-d5951d939798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:39.580379 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b3e16-dd6c-4c9f-a3bc-d5951d939798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:39.580647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b3e16-dd6c-4c9f-a3bc-d5951d939798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:39.580950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b3e16-dd6c-4c9f-a3bc-d5951d939798 admin admin] PATH_INFO: `/v3/users/f7b7f37c007a4bb39634160de8845317` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:39.582466 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b3e16-dd6c-4c9f-a3bc-d5951d939798 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7b7f37c007a4bb39634160de8845317)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:39.585252 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b3e16-dd6c-4c9f-a3bc-d5951d939798 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:39.587162 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:39.845722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-884b3e16-dd6c-4c9f-a3bc-d5951d939798 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7b7f37c007a4bb39634160de8845317'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:39.846818 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-884b3e16-dd6c-4c9f-a3bc-d5951d939798 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7b7f37c007a4bb39634160de8845317'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:39.860567 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-884b3e16-dd6c-4c9f-a3bc-d5951d939798 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7b7f37c007a4bb39634160de8845317'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:39.861593 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-884b3e16-dd6c-4c9f-a3bc-d5951d939798 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7b7f37c007a4bb39634160de8845317'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:39.938500 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-884b3e16-dd6c-4c9f-a3bc-d5951d939798 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7b7f37c007a4bb39634160de8845317'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:39.972576 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1108/2190] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:39 2022] DELETE /identity/v3/users/f7b7f37c007a4bb39634160de8845317 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:39.977919 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d5b26-b497-4ebc-97e8-04d487c572c8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:39.978412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d5b26-b497-4ebc-97e8-04d487c572c8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:39.978518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d5b26-b497-4ebc-97e8-04d487c572c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:40.000670 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-186d5b26-b497-4ebc-97e8-04d487c572c8 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:40.002021 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-186d5b26-b497-4ebc-97e8-04d487c572c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:40.014051 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:40.066038 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1083/2191] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:40.076102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b3e16-dd6c-4c9f-a3bc-d5951d939798 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:40.077545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-884b3e16-dd6c-4c9f-a3bc-d5951d939798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:40.139833 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b3e16-dd6c-4c9f-a3bc-d5951d939798 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:40.143438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f907d-09c1-43e4-9ffe-3c74ef4cc8b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:40.144085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f907d-09c1-43e4-9ffe-3c74ef4cc8b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:40.144369 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f907d-09c1-43e4-9ffe-3c74ef4cc8b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:40.150598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f907d-09c1-43e4-9ffe-3c74ef4cc8b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:40.154730 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f907d-09c1-43e4-9ffe-3c74ef4cc8b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:40.156173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f907d-09c1-43e4-9ffe-3c74ef4cc8b7 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:40.166692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1109/2192] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:40.619779 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d5b26-b497-4ebc-97e8-04d487c572c8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:40.621147 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-186d5b26-b497-4ebc-97e8-04d487c572c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:40.633964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d5b26-b497-4ebc-97e8-04d487c572c8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:40.638679 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f41f2-3883-4b9b-abb9-5da80b83f505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:40.639563 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f41f2-3883-4b9b-abb9-5da80b83f505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:40.639836 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f41f2-3883-4b9b-abb9-5da80b83f505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:40.640112 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f41f2-3883-4b9b-abb9-5da80b83f505 admin admin] PATH_INFO: `/v3/users/b0c10b946cb94e0f89674cf94e5cbc86` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:40.643348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f41f2-3883-4b9b-abb9-5da80b83f505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0c10b946cb94e0f89674cf94e5cbc86)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:40.643519 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f41f2-3883-4b9b-abb9-5da80b83f505 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:40.735464 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f907d-09c1-43e4-9ffe-3c74ef4cc8b7 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:40.737079 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ad5f907d-09c1-43e4-9ffe-3c74ef4cc8b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:40.751843 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f907d-09c1-43e4-9ffe-3c74ef4cc8b7 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:40.757922 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02358657-e1c9-4826-bfde-ef6a2d03e9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:40.758913 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02358657-e1c9-4826-bfde-ef6a2d03e9ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:40.759202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02358657-e1c9-4826-bfde-ef6a2d03e9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:40.759481 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02358657-e1c9-4826-bfde-ef6a2d03e9ef admin admin] PATH_INFO: `/v3/projects/64afb373fb4349619b34e7cfe2323bb6` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:40.760782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02358657-e1c9-4826-bfde-ef6a2d03e9ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=64afb373fb4349619b34e7cfe2323bb6)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:40.762950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02358657-e1c9-4826-bfde-ef6a2d03e9ef admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:41.031405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-887f41f2-3883-4b9b-abb9-5da80b83f505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0c10b946cb94e0f89674cf94e5cbc86'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:41.032431 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-887f41f2-3883-4b9b-abb9-5da80b83f505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0c10b946cb94e0f89674cf94e5cbc86'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:41.041112 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-887f41f2-3883-4b9b-abb9-5da80b83f505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0c10b946cb94e0f89674cf94e5cbc86'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:41.041311 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-02358657-e1c9-4826-bfde-ef6a2d03e9ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64afb373fb4349619b34e7cfe2323bb6', 'request_id': 'req-02358657-e1c9-4826-bfde-ef6a2d03e9ef'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:41.041938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-887f41f2-3883-4b9b-abb9-5da80b83f505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0c10b946cb94e0f89674cf94e5cbc86'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:41.052261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-02358657-e1c9-4826-bfde-ef6a2d03e9ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64afb373fb4349619b34e7cfe2323bb6', 'request_id': 'req-02358657-e1c9-4826-bfde-ef6a2d03e9ef'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:41.060125 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-887f41f2-3883-4b9b-abb9-5da80b83f505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0c10b946cb94e0f89674cf94e5cbc86'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:41.155551 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-02358657-e1c9-4826-bfde-ef6a2d03e9ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64afb373fb4349619b34e7cfe2323bb6', 'request_id': 'req-02358657-e1c9-4826-bfde-ef6a2d03e9ef'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:41.166770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-02358657-e1c9-4826-bfde-ef6a2d03e9ef admin admin] The token cache is being invalidate because project 64afb373fb4349619b34e7cfe2323bb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:41.167055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-02358657-e1c9-4826-bfde-ef6a2d03e9ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:41.167694 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-02358657-e1c9-4826-bfde-ef6a2d03e9ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:41.169294 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1110/2193] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:40 2022] DELETE /identity/v3/projects/64afb373fb4349619b34e7cfe2323bb6 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:39:41.174400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b65864-b07c-452a-87c6-c8fc171de0d3 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:41.174692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b65864-b07c-452a-87c6-c8fc171de0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:41.174971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b65864-b07c-452a-87c6-c8fc171de0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:41.182421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:41.194046 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-82b65864-b07c-452a-87c6-c8fc171de0d3 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:41.195359 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-82b65864-b07c-452a-87c6-c8fc171de0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:41.255489 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1111/2194] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:41.263462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae56a36-2a0f-4256-be3c-daeeb6b5391b None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:41.263633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae56a36-2a0f-4256-be3c-daeeb6b5391b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:41.263966 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae56a36-2a0f-4256-be3c-daeeb6b5391b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:41.290360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9ae56a36-2a0f-4256-be3c-daeeb6b5391b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:41.291449 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9ae56a36-2a0f-4256-be3c-daeeb6b5391b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:41.299982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1084/2195] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:40 2022] DELETE /identity/v3/users/b0c10b946cb94e0f89674cf94e5cbc86 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:41.306078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8627f-fcc3-4e62-a636-d36bb5996180 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:41.306575 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8627f-fcc3-4e62-a636-d36bb5996180 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:41.307065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8627f-fcc3-4e62-a636-d36bb5996180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:41.335441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c7c8627f-fcc3-4e62-a636-d36bb5996180 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:41.336201 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c7c8627f-fcc3-4e62-a636-d36bb5996180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:41.376838 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1112/2196] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:41.378957 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae56a36-2a0f-4256-be3c-daeeb6b5391b None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:41.381222 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9ae56a36-2a0f-4256-be3c-daeeb6b5391b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:41.396579 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae56a36-2a0f-4256-be3c-daeeb6b5391b None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:41.401624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b0c61-4e5d-46f6-8dad-45a7953b0ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:41.410715 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b0c61-4e5d-46f6-8dad-45a7953b0ee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:41.411147 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b0c61-4e5d-46f6-8dad-45a7953b0ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:41.411521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b0c61-4e5d-46f6-8dad-45a7953b0ee0 admin admin] PATH_INFO: `/v3/users/2ae9de8e4a1646b3a6bd5c51a63ecb9b` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:41.413497 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b0c61-4e5d-46f6-8dad-45a7953b0ee0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ae9de8e4a1646b3a6bd5c51a63ecb9b)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:41.416080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b0c61-4e5d-46f6-8dad-45a7953b0ee0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:41.447756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1085/2197] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:41.449525 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8627f-fcc3-4e62-a636-d36bb5996180 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:41.463028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8627f-fcc3-4e62-a636-d36bb5996180 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:41.469462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924faf52-7aa7-4728-ba7c-7acf12f97421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:41.470218 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924faf52-7aa7-4728-ba7c-7acf12f97421 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:41.470583 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924faf52-7aa7-4728-ba7c-7acf12f97421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:41.471116 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924faf52-7aa7-4728-ba7c-7acf12f97421 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:41.471772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924faf52-7aa7-4728-ba7c-7acf12f97421 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:39:41.472146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924faf52-7aa7-4728-ba7c-7acf12f97421 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:41.474658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924faf52-7aa7-4728-ba7c-7acf12f97421 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:41.487291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1086/2198] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:39:41.488167 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924faf52-7aa7-4728-ba7c-7acf12f97421 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:41.493857 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-924faf52-7aa7-4728-ba7c-7acf12f97421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:41.520259 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:41.576547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924faf52-7aa7-4728-ba7c-7acf12f97421 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:41.580618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ef9bc-3a21-4c0e-a2ff-bff9253a1b42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:41.581296 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ef9bc-3a21-4c0e-a2ff-bff9253a1b42 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:41.581577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ef9bc-3a21-4c0e-a2ff-bff9253a1b42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:41.581851 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ef9bc-3a21-4c0e-a2ff-bff9253a1b42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:41.592266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ef9bc-3a21-4c0e-a2ff-bff9253a1b42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:41.594470 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ef9bc-3a21-4c0e-a2ff-bff9253a1b42 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:41.603912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1087/2199] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:41.604757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ef9bc-3a21-4c0e-a2ff-bff9253a1b42 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:41.614574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ef9bc-3a21-4c0e-a2ff-bff9253a1b42 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:41.619591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92300f4-910d-4c97-b380-40ffbe491932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:41.620390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92300f4-910d-4c97-b380-40ffbe491932 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:41.620570 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92300f4-910d-4c97-b380-40ffbe491932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:41.620844 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92300f4-910d-4c97-b380-40ffbe491932 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:41.621845 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92300f4-910d-4c97-b380-40ffbe491932 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:41.628538 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92300f4-910d-4c97-b380-40ffbe491932 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:41.694105 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a79b0c61-4e5d-46f6-8dad-45a7953b0ee0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ae9de8e4a1646b3a6bd5c51a63ecb9b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:41.695016 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a79b0c61-4e5d-46f6-8dad-45a7953b0ee0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ae9de8e4a1646b3a6bd5c51a63ecb9b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:41.706231 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a79b0c61-4e5d-46f6-8dad-45a7953b0ee0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ae9de8e4a1646b3a6bd5c51a63ecb9b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:41.707748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a79b0c61-4e5d-46f6-8dad-45a7953b0ee0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ae9de8e4a1646b3a6bd5c51a63ecb9b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:41.741632 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1088/2200] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:41 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:41.745932 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92300f4-910d-4c97-b380-40ffbe491932 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:41.747823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d92300f4-910d-4c97-b380-40ffbe491932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:41.811374 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92300f4-910d-4c97-b380-40ffbe491932 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:41.815980 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971e07c-2fe9-47e5-8e9e-f51b7801bb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:41.816656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971e07c-2fe9-47e5-8e9e-f51b7801bb62 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:41.816942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971e07c-2fe9-47e5-8e9e-f51b7801bb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:41.818717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971e07c-2fe9-47e5-8e9e-f51b7801bb62 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:41.819128 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a79b0c61-4e5d-46f6-8dad-45a7953b0ee0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ae9de8e4a1646b3a6bd5c51a63ecb9b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:41.822499 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971e07c-2fe9-47e5-8e9e-f51b7801bb62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:41.826454 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971e07c-2fe9-47e5-8e9e-f51b7801bb62 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:41.845230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1113/2201] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:41 2022] DELETE /identity/v3/users/2ae9de8e4a1646b3a6bd5c51a63ecb9b => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:41.849727 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d7958-4af4-4d4a-ac43-2cd93e935795 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:41.850185 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d7958-4af4-4d4a-ac43-2cd93e935795 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:41.850616 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d7958-4af4-4d4a-ac43-2cd93e935795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:41.869081 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-535d7958-4af4-4d4a-ac43-2cd93e935795 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:41.870198 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-535d7958-4af4-4d4a-ac43-2cd93e935795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:41.893518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:41.917844 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1089/2202] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:41 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:41.921757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971e07c-2fe9-47e5-8e9e-f51b7801bb62 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:41.942098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971e07c-2fe9-47e5-8e9e-f51b7801bb62 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:41.972402 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1114/2203] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:41.982329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d7958-4af4-4d4a-ac43-2cd93e935795 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:41.984472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-535d7958-4af4-4d4a-ac43-2cd93e935795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:42.030934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb839b9a-bddb-47a6-b1d3-36dc9f32509b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:42.030934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb839b9a-bddb-47a6-b1d3-36dc9f32509b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:42.030934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb839b9a-bddb-47a6-b1d3-36dc9f32509b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:42.030934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb839b9a-bddb-47a6-b1d3-36dc9f32509b admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:42.031320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb839b9a-bddb-47a6-b1d3-36dc9f32509b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:42.031402 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb839b9a-bddb-47a6-b1d3-36dc9f32509b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:42.045436 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1090/2204] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:39:42.049500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb839b9a-bddb-47a6-b1d3-36dc9f32509b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:42.061604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb839b9a-bddb-47a6-b1d3-36dc9f32509b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:42.064856 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d7958-4af4-4d4a-ac43-2cd93e935795 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:42.067068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59265a-0ff3-4d90-8559-13a29bb0e8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:42.068394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59265a-0ff3-4d90-8559-13a29bb0e8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:42.068687 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59265a-0ff3-4d90-8559-13a29bb0e8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:42.068969 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59265a-0ff3-4d90-8559-13a29bb0e8e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:42.070060 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59265a-0ff3-4d90-8559-13a29bb0e8e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:42.070958 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710cd3c-4151-4ca8-999b-d3b414a60f5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:42.071694 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59265a-0ff3-4d90-8559-13a29bb0e8e5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:42.071988 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710cd3c-4151-4ca8-999b-d3b414a60f5e service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:42.072911 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710cd3c-4151-4ca8-999b-d3b414a60f5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:42.074462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710cd3c-4151-4ca8-999b-d3b414a60f5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:42.093092 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710cd3c-4151-4ca8-999b-d3b414a60f5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:42.093092 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710cd3c-4151-4ca8-999b-d3b414a60f5e service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:42.093353 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1091/2205] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:39:42.095371 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59265a-0ff3-4d90-8559-13a29bb0e8e5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:42.107046 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1115/2206] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:42.115366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59265a-0ff3-4d90-8559-13a29bb0e8e5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:42.120415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd03c7-d214-46b7-b472-b0e459c7ccec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:42.121473 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd03c7-d214-46b7-b472-b0e459c7ccec admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:42.121772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd03c7-d214-46b7-b472-b0e459c7ccec admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:42.122057 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd03c7-d214-46b7-b472-b0e459c7ccec admin admin] PATH_INFO: `/v3/projects/cf5a78ee06ad4133886cfc64aec5c908/users/792aa71311e14466a615212304864381/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:42.127370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:42.140385 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710cd3c-4151-4ca8-999b-d3b414a60f5e service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:42.140385 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6710cd3c-4151-4ca8-999b-d3b414a60f5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:42.155863 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710cd3c-4151-4ca8-999b-d3b414a60f5e service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:42.160428 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd03c7-d214-46b7-b472-b0e459c7ccec admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf5a78ee06ad4133886cfc64aec5c908, user_id=792aa71311e14466a615212304864381, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:42.172775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73756867-f4f6-43c4-bc3f-f35843be8fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:42.172775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73756867-f4f6-43c4-bc3f-f35843be8fb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:42.172775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73756867-f4f6-43c4-bc3f-f35843be8fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:42.172775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73756867-f4f6-43c4-bc3f-f35843be8fb3 admin admin] PATH_INFO: `/v3/projects/c0f861e2663f4e14827fb419a947fda7` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:42.179245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd03c7-d214-46b7-b472-b0e459c7ccec admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:42.179699 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73756867-f4f6-43c4-bc3f-f35843be8fb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0f861e2663f4e14827fb419a947fda7)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:42.181951 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73756867-f4f6-43c4-bc3f-f35843be8fb3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:42.310382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1092/2207] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:42 2022] PUT /identity/v3/projects/cf5a78ee06ad4133886cfc64aec5c908/users/792aa71311e14466a615212304864381/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:42.322298 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f46186-cc58-4c19-b674-b0658776a1b2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:42.322589 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f46186-cc58-4c19-b674-b0658776a1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:42.323055 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f46186-cc58-4c19-b674-b0658776a1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:42.362457 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c4f46186-cc58-4c19-b674-b0658776a1b2 None None] MFA Rules not processed for user `792aa71311e14466a615212304864381`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:42.363656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c4f46186-cc58-4c19-b674-b0658776a1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:42.399256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-73756867-f4f6-43c4-bc3f-f35843be8fb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0f861e2663f4e14827fb419a947fda7', 'request_id': 'req-73756867-f4f6-43c4-bc3f-f35843be8fb3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:42.410996 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-73756867-f4f6-43c4-bc3f-f35843be8fb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0f861e2663f4e14827fb419a947fda7', 'request_id': 'req-73756867-f4f6-43c4-bc3f-f35843be8fb3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:42.459588 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1093/2208] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:42.471190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f46186-cc58-4c19-b674-b0658776a1b2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:42.472924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c4f46186-cc58-4c19-b674-b0658776a1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:42.476085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-73756867-f4f6-43c4-bc3f-f35843be8fb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0f861e2663f4e14827fb419a947fda7', 'request_id': 'req-73756867-f4f6-43c4-bc3f-f35843be8fb3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:42.493049 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-73756867-f4f6-43c4-bc3f-f35843be8fb3 admin admin] The token cache is being invalidate because project c0f861e2663f4e14827fb419a947fda7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:42.493611 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-73756867-f4f6-43c4-bc3f-f35843be8fb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:42.494469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-73756867-f4f6-43c4-bc3f-f35843be8fb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:42.500551 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1116/2209] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:42 2022] DELETE /identity/v3/projects/c0f861e2663f4e14827fb419a947fda7 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:39:42.501657 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73756867-f4f6-43c4-bc3f-f35843be8fb3 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:42.506403 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-73756867-f4f6-43c4-bc3f-f35843be8fb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:42.573147 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f46186-cc58-4c19-b674-b0658776a1b2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:42.622941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73756867-f4f6-43c4-bc3f-f35843be8fb3 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:42.630541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4b4da-7a93-4512-aaff-31d3fbb645b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:42.632003 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4b4da-7a93-4512-aaff-31d3fbb645b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:42.632465 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4b4da-7a93-4512-aaff-31d3fbb645b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:42.633022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4b4da-7a93-4512-aaff-31d3fbb645b9 admin admin] PATH_INFO: `/v3/users/eabc28b6ace5413e882afb8a98d6e8fa` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:42.635071 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4b4da-7a93-4512-aaff-31d3fbb645b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eabc28b6ace5413e882afb8a98d6e8fa)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:42.636186 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39946dbf-448b-48ba-bab8-e97343e25127 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:42.641523 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39946dbf-448b-48ba-bab8-e97343e25127 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:42.641523 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39946dbf-448b-48ba-bab8-e97343e25127 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:42.641523 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39946dbf-448b-48ba-bab8-e97343e25127 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:42.641777 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4b4da-7a93-4512-aaff-31d3fbb645b9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:42.641855 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-39946dbf-448b-48ba-bab8-e97343e25127 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:42.709198 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39946dbf-448b-48ba-bab8-e97343e25127 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:42.711813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39946dbf-448b-48ba-bab8-e97343e25127 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:42.728189 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1094/2210] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:39:42.729197 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39946dbf-448b-48ba-bab8-e97343e25127 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:42.730705 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-39946dbf-448b-48ba-bab8-e97343e25127 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:42.748893 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39946dbf-448b-48ba-bab8-e97343e25127 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:42.757639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11c685-10a3-42a4-b6e2-da4524d7e58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:42.758813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11c685-10a3-42a4-b6e2-da4524d7e58d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:42.759097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11c685-10a3-42a4-b6e2-da4524d7e58d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:42.759506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11c685-10a3-42a4-b6e2-da4524d7e58d admin admin] PATH_INFO: `/v3/projects/242eef5928b14c1691d35ba2f13d7fbf` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:42.762070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11c685-10a3-42a4-b6e2-da4524d7e58d admin admin] RBAC: Authorizing `identity:delete_project(project_id=242eef5928b14c1691d35ba2f13d7fbf)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:42.764387 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11c685-10a3-42a4-b6e2-da4524d7e58d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:42.853184 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1db4b4da-7a93-4512-aaff-31d3fbb645b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eabc28b6ace5413e882afb8a98d6e8fa'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:42.854118 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1db4b4da-7a93-4512-aaff-31d3fbb645b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eabc28b6ace5413e882afb8a98d6e8fa'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:42.862478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1db4b4da-7a93-4512-aaff-31d3fbb645b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eabc28b6ace5413e882afb8a98d6e8fa'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:42.863209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1db4b4da-7a93-4512-aaff-31d3fbb645b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eabc28b6ace5413e882afb8a98d6e8fa'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:42.934244 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fc11c685-10a3-42a4-b6e2-da4524d7e58d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '242eef5928b14c1691d35ba2f13d7fbf', 'request_id': 'req-fc11c685-10a3-42a4-b6e2-da4524d7e58d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:42.941330 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fc11c685-10a3-42a4-b6e2-da4524d7e58d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '242eef5928b14c1691d35ba2f13d7fbf', 'request_id': 'req-fc11c685-10a3-42a4-b6e2-da4524d7e58d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:42.945931 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1db4b4da-7a93-4512-aaff-31d3fbb645b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eabc28b6ace5413e882afb8a98d6e8fa'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:42.948484 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fc11c685-10a3-42a4-b6e2-da4524d7e58d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '242eef5928b14c1691d35ba2f13d7fbf', 'request_id': 'req-fc11c685-10a3-42a4-b6e2-da4524d7e58d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:42.967332 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1117/2211] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:42 2022] DELETE /identity/v3/users/eabc28b6ace5413e882afb8a98d6e8fa => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:43.010556 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fc11c685-10a3-42a4-b6e2-da4524d7e58d admin admin] The token cache is being invalidate because project 242eef5928b14c1691d35ba2f13d7fbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:43.012647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fc11c685-10a3-42a4-b6e2-da4524d7e58d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:43.012647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fc11c685-10a3-42a4-b6e2-da4524d7e58d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:43.013697 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1095/2212] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:42 2022] DELETE /identity/v3/projects/242eef5928b14c1691d35ba2f13d7fbf => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:43.018667 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f70ca-567a-4bee-b409-e9be6c94fe6f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:43.019271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f70ca-567a-4bee-b409-e9be6c94fe6f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:43.019736 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f70ca-567a-4bee-b409-e9be6c94fe6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:43.041990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-271f70ca-567a-4bee-b409-e9be6c94fe6f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:43.043696 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-271f70ca-567a-4bee-b409-e9be6c94fe6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:43.114591 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1118/2213] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:43.123931 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb1d43-f968-42b6-851e-c6a6486a6866 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:43.126049 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb1d43-f968-42b6-851e-c6a6486a6866 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:43.126049 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb1d43-f968-42b6-851e-c6a6486a6866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:43.147698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c8eb1d43-f968-42b6-851e-c6a6486a6866 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:43.149053 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c8eb1d43-f968-42b6-851e-c6a6486a6866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:43.159362 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1096/2214] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:43.163825 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f70ca-567a-4bee-b409-e9be6c94fe6f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:43.183340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f70ca-567a-4bee-b409-e9be6c94fe6f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:43.189376 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50186885-688d-4e76-b762-574fbf1994cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:43.189981 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50186885-688d-4e76-b762-574fbf1994cf admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:43.190336 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50186885-688d-4e76-b762-574fbf1994cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:43.190678 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50186885-688d-4e76-b762-574fbf1994cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:43.191684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50186885-688d-4e76-b762-574fbf1994cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:39:43.191777 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50186885-688d-4e76-b762-574fbf1994cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:43.194189 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50186885-688d-4e76-b762-574fbf1994cf admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:43.207512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1119/2215] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:39:43.212398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb1d43-f968-42b6-851e-c6a6486a6866 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:43.222016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb1d43-f968-42b6-851e-c6a6486a6866 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:43.227952 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02b96d-b708-4754-a130-04fd8bcfb352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:43.228671 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02b96d-b708-4754-a130-04fd8bcfb352 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:43.228957 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02b96d-b708-4754-a130-04fd8bcfb352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:43.229274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02b96d-b708-4754-a130-04fd8bcfb352 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:43.229984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02b96d-b708-4754-a130-04fd8bcfb352 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:43.232083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02b96d-b708-4754-a130-04fd8bcfb352 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:43.259723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:43.359365 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1097/2216] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:43 2022] POST /identity/v3/projects => generated 359 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:43.363521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50186885-688d-4e76-b762-574fbf1994cf admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:43.377326 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50186885-688d-4e76-b762-574fbf1994cf admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:43.445299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02b96d-b708-4754-a130-04fd8bcfb352 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:43.448163 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92417e-9e96-47b7-8406-26d42d18ed11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:43.448972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92417e-9e96-47b7-8406-26d42d18ed11 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:43.449286 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92417e-9e96-47b7-8406-26d42d18ed11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:43.449528 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92417e-9e96-47b7-8406-26d42d18ed11 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:43.450240 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ba02b96d-b708-4754-a130-04fd8bcfb352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:43.450398 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92417e-9e96-47b7-8406-26d42d18ed11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:43.456385 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:43.457308 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92417e-9e96-47b7-8406-26d42d18ed11 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:43.496790 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02b96d-b708-4754-a130-04fd8bcfb352 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:43.507489 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82924bdd-dbbd-4619-a19c-ae85d69efd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:43.508572 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82924bdd-dbbd-4619-a19c-ae85d69efd28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:43.508862 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82924bdd-dbbd-4619-a19c-ae85d69efd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:43.509181 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82924bdd-dbbd-4619-a19c-ae85d69efd28 admin admin] PATH_INFO: `/v3/projects/d4c13e578e6e4b2981adafa571b9d197` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:43.510452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82924bdd-dbbd-4619-a19c-ae85d69efd28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4c13e578e6e4b2981adafa571b9d197)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:43.512711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82924bdd-dbbd-4619-a19c-ae85d69efd28 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:43.573824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1120/2217] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:43 2022] POST /identity/v3/users => generated 280 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:43.573824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92417e-9e96-47b7-8406-26d42d18ed11 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:43.591188 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92417e-9e96-47b7-8406-26d42d18ed11 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:43.596499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31134b5-f8c1-4122-808d-f690f92ce035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:43.597222 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31134b5-f8c1-4122-808d-f690f92ce035 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:43.597509 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31134b5-f8c1-4122-808d-f690f92ce035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:43.597805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31134b5-f8c1-4122-808d-f690f92ce035 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:43.598522 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31134b5-f8c1-4122-808d-f690f92ce035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:43.601372 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31134b5-f8c1-4122-808d-f690f92ce035 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:43.612455 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1121/2218] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:43.616760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31134b5-f8c1-4122-808d-f690f92ce035 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:43.627171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31134b5-f8c1-4122-808d-f690f92ce035 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:43.632435 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8daac-6b0b-4b1d-95c5-42cdb5b1cb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:43.633285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8daac-6b0b-4b1d-95c5-42cdb5b1cb8c admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:43.633571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8daac-6b0b-4b1d-95c5-42cdb5b1cb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:43.633850 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8daac-6b0b-4b1d-95c5-42cdb5b1cb8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:43.634617 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8daac-6b0b-4b1d-95c5-42cdb5b1cb8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:43.636305 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8daac-6b0b-4b1d-95c5-42cdb5b1cb8c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:43.651234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1122/2219] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:39:43.653531 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8daac-6b0b-4b1d-95c5-42cdb5b1cb8c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:43.670522 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8daac-6b0b-4b1d-95c5-42cdb5b1cb8c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:43.692251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-82924bdd-dbbd-4619-a19c-ae85d69efd28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4c13e578e6e4b2981adafa571b9d197', 'request_id': 'req-82924bdd-dbbd-4619-a19c-ae85d69efd28'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:43.704331 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-82924bdd-dbbd-4619-a19c-ae85d69efd28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4c13e578e6e4b2981adafa571b9d197', 'request_id': 'req-82924bdd-dbbd-4619-a19c-ae85d69efd28'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:43.721107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-82924bdd-dbbd-4619-a19c-ae85d69efd28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4c13e578e6e4b2981adafa571b9d197', 'request_id': 'req-82924bdd-dbbd-4619-a19c-ae85d69efd28'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:43.778422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc02ae4-8d69-4f10-80fa-5ea1f9036452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:43.779384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc02ae4-8d69-4f10-80fa-5ea1f9036452 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:43.779662 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc02ae4-8d69-4f10-80fa-5ea1f9036452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:43.780010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc02ae4-8d69-4f10-80fa-5ea1f9036452 admin admin] PATH_INFO: `/v3/projects/8feff0048d5b43e8b15fe2688787fb1a/users/b9d0c76b326d48648b941c778ba376fa/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:43.790928 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-82924bdd-dbbd-4619-a19c-ae85d69efd28 admin admin] The token cache is being invalidate because project d4c13e578e6e4b2981adafa571b9d197 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:43.791394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-82924bdd-dbbd-4619-a19c-ae85d69efd28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:43.792502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-82924bdd-dbbd-4619-a19c-ae85d69efd28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:43.795234 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1098/2220] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:43 2022] DELETE /identity/v3/projects/d4c13e578e6e4b2981adafa571b9d197 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:43.799275 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82924bdd-dbbd-4619-a19c-ae85d69efd28 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:43.800943 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-82924bdd-dbbd-4619-a19c-ae85d69efd28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:43.802733 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc02ae4-8d69-4f10-80fa-5ea1f9036452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8feff0048d5b43e8b15fe2688787fb1a, user_id=b9d0c76b326d48648b941c778ba376fa, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:43.805060 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc02ae4-8d69-4f10-80fa-5ea1f9036452 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:43.822494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82924bdd-dbbd-4619-a19c-ae85d69efd28 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:43.830301 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfe13a-fa82-4f76-8625-c8bf65227df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:43.831310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfe13a-fa82-4f76-8625-c8bf65227df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:43.831907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfe13a-fa82-4f76-8625-c8bf65227df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:43.832272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfe13a-fa82-4f76-8625-c8bf65227df6 admin admin] PATH_INFO: `/v3/users/6dacc2cb87094a1fb8f4e58d412ce855` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:43.834358 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bfe13a-fa82-4f76-8625-c8bf65227df6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dacc2cb87094a1fb8f4e58d412ce855)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:43.836657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bfe13a-fa82-4f76-8625-c8bf65227df6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:43.858907 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1123/2221] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:43 2022] PUT /identity/v3/projects/8feff0048d5b43e8b15fe2688787fb1a/users/b9d0c76b326d48648b941c778ba376fa/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:39:43.869375 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b11bc8-2cb2-4b4d-8c73-5dd86c75fb7c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:43.871150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b11bc8-2cb2-4b4d-8c73-5dd86c75fb7c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:43.871150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b11bc8-2cb2-4b4d-8c73-5dd86c75fb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:43.900626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-73b11bc8-2cb2-4b4d-8c73-5dd86c75fb7c None None] MFA Rules not processed for user `b9d0c76b326d48648b941c778ba376fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:43.901918 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-73b11bc8-2cb2-4b4d-8c73-5dd86c75fb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:43.965616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-13bfe13a-fa82-4f76-8625-c8bf65227df6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dacc2cb87094a1fb8f4e58d412ce855'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:43.966510 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-13bfe13a-fa82-4f76-8625-c8bf65227df6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dacc2cb87094a1fb8f4e58d412ce855'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:43.973029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1124/2222] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:43 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:43.975761 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-13bfe13a-fa82-4f76-8625-c8bf65227df6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dacc2cb87094a1fb8f4e58d412ce855'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:43.976545 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-13bfe13a-fa82-4f76-8625-c8bf65227df6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dacc2cb87094a1fb8f4e58d412ce855'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:43.989426 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-13bfe13a-fa82-4f76-8625-c8bf65227df6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dacc2cb87094a1fb8f4e58d412ce855'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:44.003616 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b11bc8-2cb2-4b4d-8c73-5dd86c75fb7c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:44.005040 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-73b11bc8-2cb2-4b4d-8c73-5dd86c75fb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:44.060743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b11bc8-2cb2-4b4d-8c73-5dd86c75fb7c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:44.064551 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac6629-976b-46cb-b068-e9d4ceddc9d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:44.065227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac6629-976b-46cb-b068-e9d4ceddc9d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:44.065525 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac6629-976b-46cb-b068-e9d4ceddc9d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:44.065782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac6629-976b-46cb-b068-e9d4ceddc9d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:44.067700 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7fac6629-976b-46cb-b068-e9d4ceddc9d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:44.077652 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1099/2223] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:43 2022] DELETE /identity/v3/users/6dacc2cb87094a1fb8f4e58d412ce855 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:39:44.081993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac6629-976b-46cb-b068-e9d4ceddc9d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:44.084117 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac6629-976b-46cb-b068-e9d4ceddc9d5 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:44.094137 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1125/2224] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:44 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:39:44.109698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfe13a-fa82-4f76-8625-c8bf65227df6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:44.111451 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-13bfe13a-fa82-4f76-8625-c8bf65227df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:44.180248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfe13a-fa82-4f76-8625-c8bf65227df6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:44.185366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c4b10-f915-44c2-b6ad-b0da1638066f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:44.186249 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c4b10-f915-44c2-b6ad-b0da1638066f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:44.186654 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c4b10-f915-44c2-b6ad-b0da1638066f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:44.186910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c4b10-f915-44c2-b6ad-b0da1638066f admin admin] PATH_INFO: `/v3/users/b9d0c76b326d48648b941c778ba376fa` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:44.188150 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c4b10-f915-44c2-b6ad-b0da1638066f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9d0c76b326d48648b941c778ba376fa)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:44.190250 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c4b10-f915-44c2-b6ad-b0da1638066f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:44.387872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-502c4b10-f915-44c2-b6ad-b0da1638066f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9d0c76b326d48648b941c778ba376fa'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:44.388550 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-502c4b10-f915-44c2-b6ad-b0da1638066f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9d0c76b326d48648b941c778ba376fa'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:44.396657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-502c4b10-f915-44c2-b6ad-b0da1638066f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9d0c76b326d48648b941c778ba376fa'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:44.397529 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-502c4b10-f915-44c2-b6ad-b0da1638066f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9d0c76b326d48648b941c778ba376fa'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:44.410601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-502c4b10-f915-44c2-b6ad-b0da1638066f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9d0c76b326d48648b941c778ba376fa'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:44.448379 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:44.482666 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1100/2225] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:44 2022] DELETE /identity/v3/users/b9d0c76b326d48648b941c778ba376fa => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:44.487527 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cb7e3-6dea-48a3-998c-56a05aa965ef None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:44.487820 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cb7e3-6dea-48a3-998c-56a05aa965ef None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:44.488105 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cb7e3-6dea-48a3-998c-56a05aa965ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:44.506892 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ce9cb7e3-6dea-48a3-998c-56a05aa965ef None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:44.512884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ce9cb7e3-6dea-48a3-998c-56a05aa965ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:44.514421 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c4b10-f915-44c2-b6ad-b0da1638066f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:44.515951 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-502c4b10-f915-44c2-b6ad-b0da1638066f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:44.615203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1126/2226] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:44.633520 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cb7e3-6dea-48a3-998c-56a05aa965ef None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:44.635834 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ce9cb7e3-6dea-48a3-998c-56a05aa965ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:44.670975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c4b10-f915-44c2-b6ad-b0da1638066f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:44.678472 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14273591-0d05-4522-a925-0803b9e0aa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:44.679755 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14273591-0d05-4522-a925-0803b9e0aa21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:44.680107 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14273591-0d05-4522-a925-0803b9e0aa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:44.680452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14273591-0d05-4522-a925-0803b9e0aa21 admin admin] PATH_INFO: `/v3/projects/8a9fddf8a3c24c2c914dc8640fcea2a3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:44.683505 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14273591-0d05-4522-a925-0803b9e0aa21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a9fddf8a3c24c2c914dc8640fcea2a3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:44.686192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14273591-0d05-4522-a925-0803b9e0aa21 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:44.727149 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cb7e3-6dea-48a3-998c-56a05aa965ef None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:44.730848 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef31cd-47d4-4fff-bdc7-8ea3d6d32e64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:44.731525 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef31cd-47d4-4fff-bdc7-8ea3d6d32e64 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:44.731867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef31cd-47d4-4fff-bdc7-8ea3d6d32e64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:44.732154 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef31cd-47d4-4fff-bdc7-8ea3d6d32e64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:44.744520 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef31cd-47d4-4fff-bdc7-8ea3d6d32e64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:44.749190 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef31cd-47d4-4fff-bdc7-8ea3d6d32e64 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:44.758003 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1127/2227] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:44.834560 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-14273591-0d05-4522-a925-0803b9e0aa21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a9fddf8a3c24c2c914dc8640fcea2a3', 'request_id': 'req-14273591-0d05-4522-a925-0803b9e0aa21'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:44.840642 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-14273591-0d05-4522-a925-0803b9e0aa21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a9fddf8a3c24c2c914dc8640fcea2a3', 'request_id': 'req-14273591-0d05-4522-a925-0803b9e0aa21'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:44.847320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-14273591-0d05-4522-a925-0803b9e0aa21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a9fddf8a3c24c2c914dc8640fcea2a3', 'request_id': 'req-14273591-0d05-4522-a925-0803b9e0aa21'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:44.898341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-14273591-0d05-4522-a925-0803b9e0aa21 admin admin] The token cache is being invalidate because project 8a9fddf8a3c24c2c914dc8640fcea2a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:44.898630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-14273591-0d05-4522-a925-0803b9e0aa21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:44.899281 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-14273591-0d05-4522-a925-0803b9e0aa21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:44.900882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1101/2228] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:44 2022] DELETE /identity/v3/projects/8a9fddf8a3c24c2c914dc8640fcea2a3 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:44.906009 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4948da4-c050-48d6-8724-04800238ac65 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:44.906485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4948da4-c050-48d6-8724-04800238ac65 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:44.906709 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4948da4-c050-48d6-8724-04800238ac65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:44.934648 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b4948da4-c050-48d6-8724-04800238ac65 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:44.936301 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b4948da4-c050-48d6-8724-04800238ac65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:44.967867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:45.052367 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1128/2229] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:45.061370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc315b9f-a042-492d-9218-9ebfd5cc01e0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:45.061674 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc315b9f-a042-492d-9218-9ebfd5cc01e0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:45.061948 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc315b9f-a042-492d-9218-9ebfd5cc01e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:45.082409 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fc315b9f-a042-492d-9218-9ebfd5cc01e0 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:45.083868 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fc315b9f-a042-492d-9218-9ebfd5cc01e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:45.092512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1102/2230] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:45.096774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4948da4-c050-48d6-8724-04800238ac65 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:45.108041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4948da4-c050-48d6-8724-04800238ac65 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:45.113329 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf889a5-7987-43b9-9846-dbe3c7e201a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:45.114129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf889a5-7987-43b9-9846-dbe3c7e201a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:45.115065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf889a5-7987-43b9-9846-dbe3c7e201a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:45.115429 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf889a5-7987-43b9-9846-dbe3c7e201a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:45.116231 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf889a5-7987-43b9-9846-dbe3c7e201a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:39:45.116600 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf889a5-7987-43b9-9846-dbe3c7e201a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:45.118474 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf889a5-7987-43b9-9846-dbe3c7e201a8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:45.140382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1129/2231] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:39:45.146703 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc315b9f-a042-492d-9218-9ebfd5cc01e0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:45.156606 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc315b9f-a042-492d-9218-9ebfd5cc01e0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:45.162748 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e33f1-a9af-4920-b343-ffe5b82bf51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:45.163428 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e33f1-a9af-4920-b343-ffe5b82bf51d admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:45.163714 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e33f1-a9af-4920-b343-ffe5b82bf51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:45.164006 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e33f1-a9af-4920-b343-ffe5b82bf51d admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:45.164687 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e33f1-a9af-4920-b343-ffe5b82bf51d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:45.166648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e33f1-a9af-4920-b343-ffe5b82bf51d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:45.193511 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf889a5-7987-43b9-9846-dbe3c7e201a8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:45.195620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-abf889a5-7987-43b9-9846-dbe3c7e201a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:45.215557 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf889a5-7987-43b9-9846-dbe3c7e201a8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:45.220791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6557544-7604-46ea-8929-c8370206a4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:45.221694 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6557544-7604-46ea-8929-c8370206a4f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:45.222032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6557544-7604-46ea-8929-c8370206a4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:45.222293 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6557544-7604-46ea-8929-c8370206a4f4 admin admin] PATH_INFO: `/v3/projects/8feff0048d5b43e8b15fe2688787fb1a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:45.223439 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6557544-7604-46ea-8929-c8370206a4f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8feff0048d5b43e8b15fe2688787fb1a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:45.225972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6557544-7604-46ea-8929-c8370206a4f4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:45.293962 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1103/2232] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:45 2022] POST /identity/v3/projects => generated 367 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:45.296696 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e33f1-a9af-4920-b343-ffe5b82bf51d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:45.359346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e33f1-a9af-4920-b343-ffe5b82bf51d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:45.363281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e6557544-7604-46ea-8929-c8370206a4f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8feff0048d5b43e8b15fe2688787fb1a', 'request_id': 'req-e6557544-7604-46ea-8929-c8370206a4f4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:45.370855 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e6557544-7604-46ea-8929-c8370206a4f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8feff0048d5b43e8b15fe2688787fb1a', 'request_id': 'req-e6557544-7604-46ea-8929-c8370206a4f4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:45.415169 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6726b-23f1-40a3-a350-742eded64b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:45.415871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6726b-23f1-40a3-a350-742eded64b62 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:45.416153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6726b-23f1-40a3-a350-742eded64b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:45.416425 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6726b-23f1-40a3-a350-742eded64b62 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:45.417227 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6726b-23f1-40a3-a350-742eded64b62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:45.419146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6726b-23f1-40a3-a350-742eded64b62 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:45.465505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e6557544-7604-46ea-8929-c8370206a4f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8feff0048d5b43e8b15fe2688787fb1a', 'request_id': 'req-e6557544-7604-46ea-8929-c8370206a4f4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:45.477043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e6557544-7604-46ea-8929-c8370206a4f4 admin admin] The token cache is being invalidate because project 8feff0048d5b43e8b15fe2688787fb1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:45.477383 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e6557544-7604-46ea-8929-c8370206a4f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:45.478038 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e6557544-7604-46ea-8929-c8370206a4f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:45.479645 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1130/2233] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:45 2022] DELETE /identity/v3/projects/8feff0048d5b43e8b15fe2688787fb1a => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:39:45.484619 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205b3e3-2874-4f42-a44c-0f3821261421 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:45.484938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205b3e3-2874-4f42-a44c-0f3821261421 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:45.485250 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205b3e3-2874-4f42-a44c-0f3821261421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:45.504635 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b205b3e3-2874-4f42-a44c-0f3821261421 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:45.505893 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b205b3e3-2874-4f42-a44c-0f3821261421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:45.514377 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1131/2234] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:45.522838 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4f418-5327-4ac4-a367-afb16fbda079 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:45.523121 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4f418-5327-4ac4-a367-afb16fbda079 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:45.523396 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4f418-5327-4ac4-a367-afb16fbda079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:45.529074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1104/2235] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:45 2022] POST /identity/v3/users => generated 284 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:45.532975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6726b-23f1-40a3-a350-742eded64b62 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:45.534910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-34a6726b-23f1-40a3-a350-742eded64b62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:45.547707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6726b-23f1-40a3-a350-742eded64b62 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:45.548919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-4da4f418-5327-4ac4-a367-afb16fbda079 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:45.550164 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4da4f418-5327-4ac4-a367-afb16fbda079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:45.559800 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a279c-5e63-44d6-96ca-b941c18fcde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:45.559800 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a279c-5e63-44d6-96ca-b941c18fcde4 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:45.559800 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a279c-5e63-44d6-96ca-b941c18fcde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:45.559800 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a279c-5e63-44d6-96ca-b941c18fcde4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:45.560252 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a279c-5e63-44d6-96ca-b941c18fcde4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:45.560252 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a279c-5e63-44d6-96ca-b941c18fcde4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:45.562395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1132/2236] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:45.567111 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4f418-5327-4ac4-a367-afb16fbda079 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:45.567853 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1105/2237] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:39:45.573423 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a279c-5e63-44d6-96ca-b941c18fcde4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:45.577093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4f418-5327-4ac4-a367-afb16fbda079 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:45.582200 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acf0a6-5264-43c1-9819-288344495f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:45.582863 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acf0a6-5264-43c1-9819-288344495f5f admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:45.583668 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acf0a6-5264-43c1-9819-288344495f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:45.583958 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acf0a6-5264-43c1-9819-288344495f5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:45.584191 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a279c-5e63-44d6-96ca-b941c18fcde4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:45.584942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acf0a6-5264-43c1-9819-288344495f5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:39:45.585304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acf0a6-5264-43c1-9819-288344495f5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:45.587037 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acf0a6-5264-43c1-9819-288344495f5f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:45.589316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702b56f-9738-4986-a4f0-7d3a646ea9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:45.593170 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702b56f-9738-4986-a4f0-7d3a646ea9ae admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:45.593170 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702b56f-9738-4986-a4f0-7d3a646ea9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:45.593170 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702b56f-9738-4986-a4f0-7d3a646ea9ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:45.593170 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702b56f-9738-4986-a4f0-7d3a646ea9ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:45.593170 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702b56f-9738-4986-a4f0-7d3a646ea9ae admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:45.602954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1133/2238] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:39:45.606173 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1106/2239] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:45.607858 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acf0a6-5264-43c1-9819-288344495f5f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:45.616416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702b56f-9738-4986-a4f0-7d3a646ea9ae admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:45.622940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acf0a6-5264-43c1-9819-288344495f5f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:45.626580 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702b56f-9738-4986-a4f0-7d3a646ea9ae admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:45.631702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25eb2f-8116-4816-a01a-78d9722ac6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:45.631988 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ef81a-049f-461c-b0f2-aae39f705a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:45.632967 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25eb2f-8116-4816-a01a-78d9722ac6b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:45.633103 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ef81a-049f-461c-b0f2-aae39f705a47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:45.633265 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ef81a-049f-461c-b0f2-aae39f705a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:45.633362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25eb2f-8116-4816-a01a-78d9722ac6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:45.633511 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ef81a-049f-461c-b0f2-aae39f705a47 admin admin] PATH_INFO: `/v3/projects/39536aa9e96c46eb8e6e62a74d5db850/users/801238c8c0c74d69bea97da67faf7141/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:45.633990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25eb2f-8116-4816-a01a-78d9722ac6b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:45.635489 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25eb2f-8116-4816-a01a-78d9722ac6b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:45.638982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25eb2f-8116-4816-a01a-78d9722ac6b0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:45.653484 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ef81a-049f-461c-b0f2-aae39f705a47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39536aa9e96c46eb8e6e62a74d5db850, user_id=801238c8c0c74d69bea97da67faf7141, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:45.656162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ef81a-049f-461c-b0f2-aae39f705a47 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:45.765427 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1134/2240] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:45 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:45.765571 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:45.769856 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25eb2f-8116-4816-a01a-78d9722ac6b0 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:45.773961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1107/2241] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:45 2022] PUT /identity/v3/projects/39536aa9e96c46eb8e6e62a74d5db850/users/801238c8c0c74d69bea97da67faf7141/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:45.786881 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc18688-fa09-4149-8736-cc137e030aba None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:45.787167 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc18688-fa09-4149-8736-cc137e030aba None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:45.787995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc18688-fa09-4149-8736-cc137e030aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:45.824836 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-acc18688-fa09-4149-8736-cc137e030aba None None] MFA Rules not processed for user `801238c8c0c74d69bea97da67faf7141`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:45.828170 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-acc18688-fa09-4149-8736-cc137e030aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:45.856232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25eb2f-8116-4816-a01a-78d9722ac6b0 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:45.861241 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325ee4f-7e9c-4496-a91f-ef007b562b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:45.863637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325ee4f-7e9c-4496-a91f-ef007b562b2a admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:45.863637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325ee4f-7e9c-4496-a91f-ef007b562b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:45.863637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325ee4f-7e9c-4496-a91f-ef007b562b2a admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:45.863637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325ee4f-7e9c-4496-a91f-ef007b562b2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:45.865209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325ee4f-7e9c-4496-a91f-ef007b562b2a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:45.904282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1108/2242] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:45 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:45.916411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc18688-fa09-4149-8736-cc137e030aba None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:45.917974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-acc18688-fa09-4149-8736-cc137e030aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:45.962676 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1135/2243] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:45 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:45.966744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325ee4f-7e9c-4496-a91f-ef007b562b2a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:45.976897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325ee4f-7e9c-4496-a91f-ef007b562b2a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:45.978526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc18688-fa09-4149-8736-cc137e030aba None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:45.982468 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff63d6a-ad95-46cd-804e-fde237779286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:45.986504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec9141-5020-4d69-96f5-352d19280469 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4d9b5e3826049f2800f1b9424977d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:45.986504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec9141-5020-4d69-96f5-352d19280469 service glance] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:45.986504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec9141-5020-4d69-96f5-352d19280469 service glance] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:45.986504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec9141-5020-4d69-96f5-352d19280469 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:45.986857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff63d6a-ad95-46cd-804e-fde237779286 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:45.986857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff63d6a-ad95-46cd-804e-fde237779286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:45.986857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff63d6a-ad95-46cd-804e-fde237779286 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:45.986857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff63d6a-ad95-46cd-804e-fde237779286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:45.988984 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff63d6a-ad95-46cd-804e-fde237779286 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:45.994907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec9141-5020-4d69-96f5-352d19280469 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:45.996547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec9141-5020-4d69-96f5-352d19280469 service glance] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:46.002916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1136/2244] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:46.006954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff63d6a-ad95-46cd-804e-fde237779286 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:46.008767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1109/2245] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:45 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:39:46.021818 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff63d6a-ad95-46cd-804e-fde237779286 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:46.041044 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf65889-8689-4bb3-81fc-7535ee122054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:46.041954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf65889-8689-4bb3-81fc-7535ee122054 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:46.042316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf65889-8689-4bb3-81fc-7535ee122054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:46.042684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf65889-8689-4bb3-81fc-7535ee122054 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:46.044140 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf65889-8689-4bb3-81fc-7535ee122054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:46.046072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf65889-8689-4bb3-81fc-7535ee122054 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:46.078165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1137/2246] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:46.081964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec9141-5020-4d69-96f5-352d19280469 service glance] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:46.095385 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec9141-5020-4d69-96f5-352d19280469 service glance] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:46.100265 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea0740-012f-4f26-8100-8c5e1e445c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:46.101242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea0740-012f-4f26-8100-8c5e1e445c20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:46.101532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea0740-012f-4f26-8100-8c5e1e445c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:46.101815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea0740-012f-4f26-8100-8c5e1e445c20 admin admin] PATH_INFO: `/v3/projects/be66894579564c8584bb7a4ff489ea22/users/489306774e664047a2a5629048455686/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:46.119134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea0740-012f-4f26-8100-8c5e1e445c20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be66894579564c8584bb7a4ff489ea22, user_id=489306774e664047a2a5629048455686, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:46.121618 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:46.121865 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea0740-012f-4f26-8100-8c5e1e445c20 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:46.206851 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1110/2247] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:46 2022] PUT /identity/v3/projects/be66894579564c8584bb7a4ff489ea22/users/489306774e664047a2a5629048455686/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:46.213336 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790b60c-828d-45da-a1c8-a1f5392f4f38 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:46.213627 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790b60c-828d-45da-a1c8-a1f5392f4f38 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:46.213912 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790b60c-828d-45da-a1c8-a1f5392f4f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:46.250735 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-3790b60c-828d-45da-a1c8-a1f5392f4f38 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:46.252187 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3790b60c-828d-45da-a1c8-a1f5392f4f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:46.328824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1138/2248] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:39:46.337221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea0740-012f-4f26-8100-8c5e1e445c20 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:46.340597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-61ea0740-012f-4f26-8100-8c5e1e445c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:46.398041 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea0740-012f-4f26-8100-8c5e1e445c20 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:46.401947 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6faef1e-4c25-4035-afeb-e77b8099304f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:46.402598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6faef1e-4c25-4035-afeb-e77b8099304f service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:46.402878 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6faef1e-4c25-4035-afeb-e77b8099304f service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:46.403152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6faef1e-4c25-4035-afeb-e77b8099304f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:46.413509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6faef1e-4c25-4035-afeb-e77b8099304f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:46.415037 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6faef1e-4c25-4035-afeb-e77b8099304f service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:46.424740 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1111/2249] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:46.493504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:46.538788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:46.809827 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:47.373553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:47.762363 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524b466-7a86-4b59-b6df-a17cfdf3d516 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:47.762363 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524b466-7a86-4b59-b6df-a17cfdf3d516 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:47.762363 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524b466-7a86-4b59-b6df-a17cfdf3d516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:47.787863 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b524b466-7a86-4b59-b6df-a17cfdf3d516 None None] MFA Rules not processed for user `489306774e664047a2a5629048455686`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:47.789623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b524b466-7a86-4b59-b6df-a17cfdf3d516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:47.899397 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1139/2250] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:47.912869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6faef1e-4c25-4035-afeb-e77b8099304f service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:47.912869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b6faef1e-4c25-4035-afeb-e77b8099304f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:47.980667 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6faef1e-4c25-4035-afeb-e77b8099304f service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:47.986383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae1d2a-1a79-4116-b082-78002c0fa41e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:47.987062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae1d2a-1a79-4116-b082-78002c0fa41e service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:47.987346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae1d2a-1a79-4116-b082-78002c0fa41e service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:47.987624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae1d2a-1a79-4116-b082-78002c0fa41e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:47.997649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae1d2a-1a79-4116-b082-78002c0fa41e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:47.999234 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae1d2a-1a79-4116-b082-78002c0fa41e service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:48.011074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1112/2251] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:49.227528 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524b466-7a86-4b59-b6df-a17cfdf3d516 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:49.229527 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b524b466-7a86-4b59-b6df-a17cfdf3d516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:49.250382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524b466-7a86-4b59-b6df-a17cfdf3d516 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:49.257578 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d47e9f-f822-4471-bafe-2f1054b65b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:49.259995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d47e9f-f822-4471-bafe-2f1054b65b6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:49.259995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d47e9f-f822-4471-bafe-2f1054b65b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:49.259995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d47e9f-f822-4471-bafe-2f1054b65b6b admin admin] PATH_INFO: `/v3/users/792aa71311e14466a615212304864381` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:49.261356 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d47e9f-f822-4471-bafe-2f1054b65b6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=792aa71311e14466a615212304864381)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:49.263801 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d47e9f-f822-4471-bafe-2f1054b65b6b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:49.422929 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-62d47e9f-f822-4471-bafe-2f1054b65b6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '792aa71311e14466a615212304864381'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:49.423714 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-62d47e9f-f822-4471-bafe-2f1054b65b6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '792aa71311e14466a615212304864381'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:49.431812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-62d47e9f-f822-4471-bafe-2f1054b65b6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '792aa71311e14466a615212304864381'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:49.432500 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-62d47e9f-f822-4471-bafe-2f1054b65b6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '792aa71311e14466a615212304864381'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:49.504940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-62d47e9f-f822-4471-bafe-2f1054b65b6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '792aa71311e14466a615212304864381'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:49.525305 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1140/2252] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:49 2022] DELETE /identity/v3/users/792aa71311e14466a615212304864381 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:49.530525 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3803b0f-93bf-4858-abe5-1c82a6930c6d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:49.531072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3803b0f-93bf-4858-abe5-1c82a6930c6d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:49.531163 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3803b0f-93bf-4858-abe5-1c82a6930c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:49.553238 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b3803b0f-93bf-4858-abe5-1c82a6930c6d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:49.553628 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b3803b0f-93bf-4858-abe5-1c82a6930c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:49.621171 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1113/2253] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:49.637298 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d47e9f-f822-4471-bafe-2f1054b65b6b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:49.638741 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-62d47e9f-f822-4471-bafe-2f1054b65b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:49.651034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:49.699489 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d47e9f-f822-4471-bafe-2f1054b65b6b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:49.703677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d65807-cd82-4968-b58f-315bb468bda5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:49.704353 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d65807-cd82-4968-b58f-315bb468bda5 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:49.704640 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d65807-cd82-4968-b58f-315bb468bda5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:49.704929 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d65807-cd82-4968-b58f-315bb468bda5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:49.715334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d65807-cd82-4968-b58f-315bb468bda5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:49.716799 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d65807-cd82-4968-b58f-315bb468bda5 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:49.727006 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1141/2254] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:49.819723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed48777-aae1-4bdd-8d5e-eff51e5a09a2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:49.820482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed48777-aae1-4bdd-8d5e-eff51e5a09a2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:49.824729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed48777-aae1-4bdd-8d5e-eff51e5a09a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:49.842304 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bed48777-aae1-4bdd-8d5e-eff51e5a09a2 None None] MFA Rules not processed for user `489306774e664047a2a5629048455686`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:49.843576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bed48777-aae1-4bdd-8d5e-eff51e5a09a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:49.858057 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:49.910310 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1114/2255] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:50.356036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d65807-cd82-4968-b58f-315bb468bda5 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:50.358111 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-55d65807-cd82-4968-b58f-315bb468bda5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:50.398724 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d65807-cd82-4968-b58f-315bb468bda5 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:50.405093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253509b-34ef-4150-9a56-b01ccb206e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:50.407673 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253509b-34ef-4150-9a56-b01ccb206e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:50.407998 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253509b-34ef-4150-9a56-b01ccb206e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:50.408530 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253509b-34ef-4150-9a56-b01ccb206e9a admin admin] PATH_INFO: `/v3/projects/cf5a78ee06ad4133886cfc64aec5c908` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:50.410157 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253509b-34ef-4150-9a56-b01ccb206e9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf5a78ee06ad4133886cfc64aec5c908)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:50.412399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253509b-34ef-4150-9a56-b01ccb206e9a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:50.570313 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f253509b-34ef-4150-9a56-b01ccb206e9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf5a78ee06ad4133886cfc64aec5c908', 'request_id': 'req-f253509b-34ef-4150-9a56-b01ccb206e9a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:50.584238 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f253509b-34ef-4150-9a56-b01ccb206e9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf5a78ee06ad4133886cfc64aec5c908', 'request_id': 'req-f253509b-34ef-4150-9a56-b01ccb206e9a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:50.659645 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f253509b-34ef-4150-9a56-b01ccb206e9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf5a78ee06ad4133886cfc64aec5c908', 'request_id': 'req-f253509b-34ef-4150-9a56-b01ccb206e9a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:50.679740 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f253509b-34ef-4150-9a56-b01ccb206e9a admin admin] The token cache is being invalidate because project cf5a78ee06ad4133886cfc64aec5c908 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:50.680027 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f253509b-34ef-4150-9a56-b01ccb206e9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:50.680772 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f253509b-34ef-4150-9a56-b01ccb206e9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:50.682597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1142/2256] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:50 2022] DELETE /identity/v3/projects/cf5a78ee06ad4133886cfc64aec5c908 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:50.687303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03414ac-bb04-497f-a4e6-75249fd33901 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:50.687845 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03414ac-bb04-497f-a4e6-75249fd33901 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:50.688070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03414ac-bb04-497f-a4e6-75249fd33901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:50.711907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d03414ac-bb04-497f-a4e6-75249fd33901 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:50.713173 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d03414ac-bb04-497f-a4e6-75249fd33901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:50.814891 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1115/2257] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:50.831978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7305bf-28e3-4a81-994f-a617d11efe83 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:50.833312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7305bf-28e3-4a81-994f-a617d11efe83 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:50.833863 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7305bf-28e3-4a81-994f-a617d11efe83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:50.868444 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ce7305bf-28e3-4a81-994f-a617d11efe83 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:50.870282 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ce7305bf-28e3-4a81-994f-a617d11efe83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:50.889751 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1143/2258] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:50.896193 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03414ac-bb04-497f-a4e6-75249fd33901 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:50.915166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03414ac-bb04-497f-a4e6-75249fd33901 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:50.921650 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc671e4-f0ab-42dc-a433-b9d35fe3a611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:50.922515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc671e4-f0ab-42dc-a433-b9d35fe3a611 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:50.924552 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc671e4-f0ab-42dc-a433-b9d35fe3a611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:50.924552 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc671e4-f0ab-42dc-a433-b9d35fe3a611 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:50.924552 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc671e4-f0ab-42dc-a433-b9d35fe3a611 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:39:50.924552 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc671e4-f0ab-42dc-a433-b9d35fe3a611 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:50.925818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc671e4-f0ab-42dc-a433-b9d35fe3a611 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:50.945129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1116/2259] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:39:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:39:50.947165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7305bf-28e3-4a81-994f-a617d11efe83 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:50.963872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7305bf-28e3-4a81-994f-a617d11efe83 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:50.980689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920b79b-6e7e-4106-badc-4571757f3bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:50.980689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920b79b-6e7e-4106-badc-4571757f3bbf admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:50.980689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920b79b-6e7e-4106-badc-4571757f3bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:50.980689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920b79b-6e7e-4106-badc-4571757f3bbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:50.981470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e920b79b-6e7e-4106-badc-4571757f3bbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:50.981470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e920b79b-6e7e-4106-badc-4571757f3bbf admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:51.100950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1144/2260] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:50 2022] POST /identity/v3/projects => generated 363 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:51.106800 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc671e4-f0ab-42dc-a433-b9d35fe3a611 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:51.119882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc671e4-f0ab-42dc-a433-b9d35fe3a611 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:51.225357 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1c612-e3a7-40ee-b6ca-4e7018a4c91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:51.226066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1c612-e3a7-40ee-b6ca-4e7018a4c91a admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:51.226414 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1c612-e3a7-40ee-b6ca-4e7018a4c91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:51.226807 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1c612-e3a7-40ee-b6ca-4e7018a4c91a admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:51.227493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1c612-e3a7-40ee-b6ca-4e7018a4c91a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:51.229549 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1c612-e3a7-40ee-b6ca-4e7018a4c91a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:51.351608 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1117/2261] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:51 2022] POST /identity/v3/users => generated 282 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:39:51.355748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920b79b-6e7e-4106-badc-4571757f3bbf admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:51.367070 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920b79b-6e7e-4106-badc-4571757f3bbf admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:51.373769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a88fd-b147-416c-8cf3-08f02efcb2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:51.374820 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a88fd-b147-416c-8cf3-08f02efcb2f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:51.375260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a88fd-b147-416c-8cf3-08f02efcb2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:51.375717 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a88fd-b147-416c-8cf3-08f02efcb2f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:51.377021 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a88fd-b147-416c-8cf3-08f02efcb2f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:51.378867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a88fd-b147-416c-8cf3-08f02efcb2f0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:51.399907 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1145/2262] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:39:51.403843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1c612-e3a7-40ee-b6ca-4e7018a4c91a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:51.414373 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1c612-e3a7-40ee-b6ca-4e7018a4c91a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:51.419369 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2e642-6106-46b9-a2b9-76081262de61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:51.420030 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2e642-6106-46b9-a2b9-76081262de61 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:51.420318 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2e642-6106-46b9-a2b9-76081262de61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:51.420596 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2e642-6106-46b9-a2b9-76081262de61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:51.421359 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2e642-6106-46b9-a2b9-76081262de61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:51.423163 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2e642-6106-46b9-a2b9-76081262de61 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:51.435372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1118/2263] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:51.439753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a88fd-b147-416c-8cf3-08f02efcb2f0 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:51.462244 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a88fd-b147-416c-8cf3-08f02efcb2f0 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:51.467283 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ea6ac-87cd-43af-b597-d3770e7bbbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:51.468389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ea6ac-87cd-43af-b597-d3770e7bbbe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:51.468692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ea6ac-87cd-43af-b597-d3770e7bbbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:51.469258 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ea6ac-87cd-43af-b597-d3770e7bbbe1 admin admin] PATH_INFO: `/v3/projects/53f3a1dc6bbf4cd382c7f507fed85049/users/f9945de895484b66acf146b131043f40/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:51.485953 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ea6ac-87cd-43af-b597-d3770e7bbbe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53f3a1dc6bbf4cd382c7f507fed85049, user_id=f9945de895484b66acf146b131043f40, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:51.488636 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ea6ac-87cd-43af-b597-d3770e7bbbe1 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:51.629673 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1146/2264] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:51 2022] PUT /identity/v3/projects/53f3a1dc6bbf4cd382c7f507fed85049/users/f9945de895484b66acf146b131043f40/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:51.640888 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b146f-6669-47b8-9113-0833011307a4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:51.641216 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b146f-6669-47b8-9113-0833011307a4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:51.641577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b146f-6669-47b8-9113-0833011307a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:51.672858 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-751b146f-6669-47b8-9113-0833011307a4 None None] MFA Rules not processed for user `f9945de895484b66acf146b131043f40`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:51.674230 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-751b146f-6669-47b8-9113-0833011307a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:51.696657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:51.746590 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1119/2265] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:51 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:51.759420 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ea6ac-87cd-43af-b597-d3770e7bbbe1 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:51.761750 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-976ea6ac-87cd-43af-b597-d3770e7bbbe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:51.805074 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:51.843257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ea6ac-87cd-43af-b597-d3770e7bbbe1 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:51.851351 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4218f43-e65b-40a7-afb2-3507a15a2b4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:51.852028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4218f43-e65b-40a7-afb2-3507a15a2b4d service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:51.852310 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4218f43-e65b-40a7-afb2-3507a15a2b4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:51.852598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4218f43-e65b-40a7-afb2-3507a15a2b4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:51.870974 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4218f43-e65b-40a7-afb2-3507a15a2b4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:51.870974 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4218f43-e65b-40a7-afb2-3507a15a2b4d service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:51.892106 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1147/2266] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:51 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:52.653657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b146f-6669-47b8-9113-0833011307a4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:52.658500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-751b146f-6669-47b8-9113-0833011307a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:52.776579 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b146f-6669-47b8-9113-0833011307a4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:52.781399 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a3254-188a-48d0-aab1-01a4d0a5ac5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:52.782420 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a3254-188a-48d0-aab1-01a4d0a5ac5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:52.782767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a3254-188a-48d0-aab1-01a4d0a5ac5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:52.783111 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a3254-188a-48d0-aab1-01a4d0a5ac5a admin admin] PATH_INFO: `/v3/users/45c0423c96764cf2a7706dc7a56bc1d5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:52.784671 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a3254-188a-48d0-aab1-01a4d0a5ac5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c0423c96764cf2a7706dc7a56bc1d5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:52.786796 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a3254-188a-48d0-aab1-01a4d0a5ac5a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:52.972533 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cb8a3254-188a-48d0-aab1-01a4d0a5ac5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c0423c96764cf2a7706dc7a56bc1d5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:52.973303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cb8a3254-188a-48d0-aab1-01a4d0a5ac5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c0423c96764cf2a7706dc7a56bc1d5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:52.981090 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cb8a3254-188a-48d0-aab1-01a4d0a5ac5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c0423c96764cf2a7706dc7a56bc1d5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:52.981798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cb8a3254-188a-48d0-aab1-01a4d0a5ac5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c0423c96764cf2a7706dc7a56bc1d5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:52.993615 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cb8a3254-188a-48d0-aab1-01a4d0a5ac5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c0423c96764cf2a7706dc7a56bc1d5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:53.128506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1120/2267] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:52 2022] DELETE /identity/v3/users/45c0423c96764cf2a7706dc7a56bc1d5 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:53.133231 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea2244-5f6a-4bd0-9746-5933881b6d97 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:53.133730 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea2244-5f6a-4bd0-9746-5933881b6d97 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:53.134034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea2244-5f6a-4bd0-9746-5933881b6d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:53.154331 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-65ea2244-5f6a-4bd0-9746-5933881b6d97 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:53.155895 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-65ea2244-5f6a-4bd0-9746-5933881b6d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:53.240942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1148/2268] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:53.251159 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a3254-188a-48d0-aab1-01a4d0a5ac5a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:53.252367 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cb8a3254-188a-48d0-aab1-01a4d0a5ac5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:53.319758 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:53.335870 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a3254-188a-48d0-aab1-01a4d0a5ac5a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:53.339778 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076e69d-01be-4607-91e3-041caf44f68d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:53.340436 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076e69d-01be-4607-91e3-041caf44f68d service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:53.340715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076e69d-01be-4607-91e3-041caf44f68d service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:53.340989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076e69d-01be-4607-91e3-041caf44f68d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:53.354831 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076e69d-01be-4607-91e3-041caf44f68d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:53.356795 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076e69d-01be-4607-91e3-041caf44f68d service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:53.370563 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1121/2269] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:54.148867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea2244-5f6a-4bd0-9746-5933881b6d97 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:54.150985 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-65ea2244-5f6a-4bd0-9746-5933881b6d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:54.194577 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea2244-5f6a-4bd0-9746-5933881b6d97 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:54.206992 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f1658-4892-422a-af9c-d1e3cb3a0ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:54.208374 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f1658-4892-422a-af9c-d1e3cb3a0ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:54.209712 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f1658-4892-422a-af9c-d1e3cb3a0ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:54.210144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f1658-4892-422a-af9c-d1e3cb3a0ed6 admin admin] PATH_INFO: `/v3/projects/00a507ce611e4d3dbc43a79811cb55a9` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:54.218014 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f1658-4892-422a-af9c-d1e3cb3a0ed6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00a507ce611e4d3dbc43a79811cb55a9)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:54.221137 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f1658-4892-422a-af9c-d1e3cb3a0ed6 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:54.382320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6a3f1658-4892-422a-af9c-d1e3cb3a0ed6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00a507ce611e4d3dbc43a79811cb55a9', 'request_id': 'req-6a3f1658-4892-422a-af9c-d1e3cb3a0ed6'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:54.393654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6a3f1658-4892-422a-af9c-d1e3cb3a0ed6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00a507ce611e4d3dbc43a79811cb55a9', 'request_id': 'req-6a3f1658-4892-422a-af9c-d1e3cb3a0ed6'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:54.493801 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6a3f1658-4892-422a-af9c-d1e3cb3a0ed6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00a507ce611e4d3dbc43a79811cb55a9', 'request_id': 'req-6a3f1658-4892-422a-af9c-d1e3cb3a0ed6'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:54.507180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6a3f1658-4892-422a-af9c-d1e3cb3a0ed6 admin admin] The token cache is being invalidate because project 00a507ce611e4d3dbc43a79811cb55a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:54.507821 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6a3f1658-4892-422a-af9c-d1e3cb3a0ed6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:54.509015 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6a3f1658-4892-422a-af9c-d1e3cb3a0ed6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:54.510755 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1149/2270] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:54 2022] DELETE /identity/v3/projects/00a507ce611e4d3dbc43a79811cb55a9 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:54.515419 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076e69d-01be-4607-91e3-041caf44f68d service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:54.527100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076e69d-01be-4607-91e3-041caf44f68d service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:54.634548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2e2b8-2458-4ae8-8eaf-de2d50448027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:54.635457 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2e2b8-2458-4ae8-8eaf-de2d50448027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:54.635886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2e2b8-2458-4ae8-8eaf-de2d50448027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:54.636246 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2e2b8-2458-4ae8-8eaf-de2d50448027 admin admin] PATH_INFO: `/v3/users/b0dfc1322cbe4f8086174f0694016710` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:54.637595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2e2b8-2458-4ae8-8eaf-de2d50448027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0dfc1322cbe4f8086174f0694016710)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:54.639809 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2e2b8-2458-4ae8-8eaf-de2d50448027 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:54.865788 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ead2e2b8-2458-4ae8-8eaf-de2d50448027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0dfc1322cbe4f8086174f0694016710'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:54.865788 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ead2e2b8-2458-4ae8-8eaf-de2d50448027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0dfc1322cbe4f8086174f0694016710'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:54.877772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ead2e2b8-2458-4ae8-8eaf-de2d50448027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0dfc1322cbe4f8086174f0694016710'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:54.878734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ead2e2b8-2458-4ae8-8eaf-de2d50448027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0dfc1322cbe4f8086174f0694016710'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:54.898934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ead2e2b8-2458-4ae8-8eaf-de2d50448027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0dfc1322cbe4f8086174f0694016710'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:54.994940 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1122/2271] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:54 2022] DELETE /identity/v3/users/b0dfc1322cbe4f8086174f0694016710 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:55.649557 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f1658-4892-422a-af9c-d1e3cb3a0ed6 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:55.651768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6a3f1658-4892-422a-af9c-d1e3cb3a0ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:55.666118 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f1658-4892-422a-af9c-d1e3cb3a0ed6 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:55.725191 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:55.731404 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb9150-80e4-4df7-be1b-51617c132f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:55.732278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb9150-80e4-4df7-be1b-51617c132f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:55.732564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb9150-80e4-4df7-be1b-51617c132f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:55.732844 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb9150-80e4-4df7-be1b-51617c132f1f admin admin] PATH_INFO: `/v3/projects/13ecc7f298a84c5f9282d6ca5bd9b972` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:55.734133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb9150-80e4-4df7-be1b-51617c132f1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=13ecc7f298a84c5f9282d6ca5bd9b972)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:55.736066 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb9150-80e4-4df7-be1b-51617c132f1f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:55.860326 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b2bb9150-80e4-4df7-be1b-51617c132f1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13ecc7f298a84c5f9282d6ca5bd9b972', 'request_id': 'req-b2bb9150-80e4-4df7-be1b-51617c132f1f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:55.866494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b2bb9150-80e4-4df7-be1b-51617c132f1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13ecc7f298a84c5f9282d6ca5bd9b972', 'request_id': 'req-b2bb9150-80e4-4df7-be1b-51617c132f1f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:55.992838 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b2bb9150-80e4-4df7-be1b-51617c132f1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13ecc7f298a84c5f9282d6ca5bd9b972', 'request_id': 'req-b2bb9150-80e4-4df7-be1b-51617c132f1f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:56.006797 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b2bb9150-80e4-4df7-be1b-51617c132f1f admin admin] The token cache is being invalidate because project 13ecc7f298a84c5f9282d6ca5bd9b972 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:39:56.007187 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b2bb9150-80e4-4df7-be1b-51617c132f1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:56.007963 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b2bb9150-80e4-4df7-be1b-51617c132f1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:56.009657 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1150/2272] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:39:55 2022] DELETE /identity/v3/projects/13ecc7f298a84c5f9282d6ca5bd9b972 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:39:56.014893 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26399a5e-b2c6-4ab5-ad92-3e55e71ef221 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:56.016092 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26399a5e-b2c6-4ab5-ad92-3e55e71ef221 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:56.016882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26399a5e-b2c6-4ab5-ad92-3e55e71ef221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:56.057373 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-26399a5e-b2c6-4ab5-ad92-3e55e71ef221 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:56.058636 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-26399a5e-b2c6-4ab5-ad92-3e55e71ef221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:56.141330 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1123/2273] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:56.152443 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cc407-066b-442a-aa2e-62ac4f382b49 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:56.152734 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cc407-066b-442a-aa2e-62ac4f382b49 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:56.153029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cc407-066b-442a-aa2e-62ac4f382b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:56.195201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a09cc407-066b-442a-aa2e-62ac4f382b49 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:56.197829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a09cc407-066b-442a-aa2e-62ac4f382b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:56.222713 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1151/2274] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:56.225890 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26399a5e-b2c6-4ab5-ad92-3e55e71ef221 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:56.234206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:56.240927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26399a5e-b2c6-4ab5-ad92-3e55e71ef221 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:56.247544 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb53c5e-17a7-457b-b424-05a4a6ab0a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:56.248545 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb53c5e-17a7-457b-b424-05a4a6ab0a69 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:56.250890 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb53c5e-17a7-457b-b424-05a4a6ab0a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:56.250890 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb53c5e-17a7-457b-b424-05a4a6ab0a69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:56.252867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb53c5e-17a7-457b-b424-05a4a6ab0a69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:39:56.253561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb53c5e-17a7-457b-b424-05a4a6ab0a69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:56.256706 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb53c5e-17a7-457b-b424-05a4a6ab0a69 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:56.272624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1124/2275] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:39:56.279092 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09cc407-066b-442a-aa2e-62ac4f382b49 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:56.297861 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09cc407-066b-442a-aa2e-62ac4f382b49 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:56.303773 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9e02d-a0ba-4ec5-baec-e7b792155ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:56.304753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9e02d-a0ba-4ec5-baec-e7b792155ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:56.305409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9e02d-a0ba-4ec5-baec-e7b792155ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:56.305892 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9e02d-a0ba-4ec5-baec-e7b792155ee9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:56.308406 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9e02d-a0ba-4ec5-baec-e7b792155ee9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:56.312954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9e02d-a0ba-4ec5-baec-e7b792155ee9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:56.483949 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1152/2276] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:56 2022] POST /identity/v3/projects => generated 383 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:56.489330 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb53c5e-17a7-457b-b424-05a4a6ab0a69 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:56.533168 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb53c5e-17a7-457b-b424-05a4a6ab0a69 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:56.683151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ce71c-0a71-44a8-845d-3a4e53a228b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:56.683866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ce71c-0a71-44a8-845d-3a4e53a228b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:56.684148 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ce71c-0a71-44a8-845d-3a4e53a228b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:56.686190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ce71c-0a71-44a8-845d-3a4e53a228b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:56.686190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ce71c-0a71-44a8-845d-3a4e53a228b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:56.688224 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ce71c-0a71-44a8-845d-3a4e53a228b3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:56.801550 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1125/2277] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:56 2022] POST /identity/v3/users => generated 292 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:56.802542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9e02d-a0ba-4ec5-baec-e7b792155ee9 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:56.813450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9e02d-a0ba-4ec5-baec-e7b792155ee9 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:56.818468 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a86db-fb1f-41f7-97dc-d53dcf3b36a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:56.819246 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a86db-fb1f-41f7-97dc-d53dcf3b36a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:56.819580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a86db-fb1f-41f7-97dc-d53dcf3b36a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:56.820204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a86db-fb1f-41f7-97dc-d53dcf3b36a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:56.821032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a86db-fb1f-41f7-97dc-d53dcf3b36a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:56.822883 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a86db-fb1f-41f7-97dc-d53dcf3b36a0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:56.835919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1153/2278] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:56.839650 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ce71c-0a71-44a8-845d-3a4e53a228b3 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:56.852295 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ce71c-0a71-44a8-845d-3a4e53a228b3 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:56.857139 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b012401-7827-479d-b766-51602ed6cd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:56.857818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b012401-7827-479d-b766-51602ed6cd28 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:56.858094 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b012401-7827-479d-b766-51602ed6cd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:56.858377 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b012401-7827-479d-b766-51602ed6cd28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:56.859111 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b012401-7827-479d-b766-51602ed6cd28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:56.860873 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b012401-7827-479d-b766-51602ed6cd28 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:56.875851 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1126/2279] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:56.880944 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a86db-fb1f-41f7-97dc-d53dcf3b36a0 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:56.897052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a86db-fb1f-41f7-97dc-d53dcf3b36a0 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:56.902980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6bf09-e8a5-4f5c-b7a0-d50de5e75554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:56.904017 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6bf09-e8a5-4f5c-b7a0-d50de5e75554 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:56.904396 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6bf09-e8a5-4f5c-b7a0-d50de5e75554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:56.904740 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6bf09-e8a5-4f5c-b7a0-d50de5e75554 admin admin] PATH_INFO: `/v3/projects/110e8b0206ec41b4a6bd8f044b3f44e6/users/2fab6a3a0aee40fbb0310c8bbc39c0a2/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:56.921930 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6bf09-e8a5-4f5c-b7a0-d50de5e75554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=110e8b0206ec41b4a6bd8f044b3f44e6, user_id=2fab6a3a0aee40fbb0310c8bbc39c0a2, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:56.924832 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6bf09-e8a5-4f5c-b7a0-d50de5e75554 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:56.997769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1154/2280] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:56 2022] PUT /identity/v3/projects/110e8b0206ec41b4a6bd8f044b3f44e6/users/2fab6a3a0aee40fbb0310c8bbc39c0a2/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:57.009644 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff7ecd-2dc4-492c-829f-b222917cffd6 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:57.009934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff7ecd-2dc4-492c-829f-b222917cffd6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:57.010216 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff7ecd-2dc4-492c-829f-b222917cffd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:57.038804 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-03ff7ecd-2dc4-492c-829f-b222917cffd6 None None] MFA Rules not processed for user `2fab6a3a0aee40fbb0310c8bbc39c0a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:57.040040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-03ff7ecd-2dc4-492c-829f-b222917cffd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:57.114093 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1127/2281] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:57.118950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6bf09-e8a5-4f5c-b7a0-d50de5e75554 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:57.185304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6bf09-e8a5-4f5c-b7a0-d50de5e75554 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:57.190529 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab02262-727f-4023-8b36-3b7d39694aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:57.191415 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab02262-727f-4023-8b36-3b7d39694aca admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:57.191792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab02262-727f-4023-8b36-3b7d39694aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:57.191984 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab02262-727f-4023-8b36-3b7d39694aca admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:57.193489 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab02262-727f-4023-8b36-3b7d39694aca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:57.195611 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab02262-727f-4023-8b36-3b7d39694aca admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:57.446947 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1155/2282] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:39:57 2022] POST /identity/v3/projects => generated 381 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:57.451309 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff7ecd-2dc4-492c-829f-b222917cffd6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:57.463380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff7ecd-2dc4-492c-829f-b222917cffd6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:57.523777 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:57.532276 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5d15c-83c4-4d2c-b26a-73352d11f3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:57.533024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5d15c-83c4-4d2c-b26a-73352d11f3e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:57.533669 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5d15c-83c4-4d2c-b26a-73352d11f3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:57.533955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5d15c-83c4-4d2c-b26a-73352d11f3e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:57.534690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb5d15c-83c4-4d2c-b26a-73352d11f3e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:57.537453 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb5d15c-83c4-4d2c-b26a-73352d11f3e5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:57.666129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1128/2283] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:39:57 2022] POST /identity/v3/users => generated 297 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:39:57.668430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab02262-727f-4023-8b36-3b7d39694aca admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:57.680127 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab02262-727f-4023-8b36-3b7d39694aca admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:57.688688 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6bc05-cb72-4f75-b834-dc27cd88878f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:57.689528 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6bc05-cb72-4f75-b834-dc27cd88878f admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:57.689819 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6bc05-cb72-4f75-b834-dc27cd88878f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:57.690085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6bc05-cb72-4f75-b834-dc27cd88878f admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:57.690844 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6bc05-cb72-4f75-b834-dc27cd88878f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:57.692556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6bc05-cb72-4f75-b834-dc27cd88878f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:57.695374 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:57.704691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1156/2284] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:57.709673 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5d15c-83c4-4d2c-b26a-73352d11f3e5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:57.719990 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5d15c-83c4-4d2c-b26a-73352d11f3e5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:57.725525 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659598c-f478-406c-ac2d-74b553920c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:57.726413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659598c-f478-406c-ac2d-74b553920c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:57.726692 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659598c-f478-406c-ac2d-74b553920c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:57.726964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659598c-f478-406c-ac2d-74b553920c04 admin admin] PATH_INFO: `/v3/projects/7420f594761449f1b59fc82dfea048b6/users/e2d38daf67f84ae09394628b2a2de74c/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:57.743548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659598c-f478-406c-ac2d-74b553920c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7420f594761449f1b59fc82dfea048b6, user_id=e2d38daf67f84ae09394628b2a2de74c, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:57.746253 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659598c-f478-406c-ac2d-74b553920c04 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:57.821532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1129/2285] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:57 2022] PUT /identity/v3/projects/7420f594761449f1b59fc82dfea048b6/users/e2d38daf67f84ae09394628b2a2de74c/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:39:57.825405 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6bc05-cb72-4f75-b834-dc27cd88878f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:57.836399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6bc05-cb72-4f75-b834-dc27cd88878f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:57.900543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf61f20-d780-4578-bd6b-df3ad6312e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:57.901262 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf61f20-d780-4578-bd6b-df3ad6312e43 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:57.901519 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf61f20-d780-4578-bd6b-df3ad6312e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:57.901793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf61f20-d780-4578-bd6b-df3ad6312e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:57.903605 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf61f20-d780-4578-bd6b-df3ad6312e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:57.904368 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf61f20-d780-4578-bd6b-df3ad6312e43 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:57.920723 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1157/2286] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:39:57.922001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659598c-f478-406c-ac2d-74b553920c04 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:57.937375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659598c-f478-406c-ac2d-74b553920c04 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:57.951045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e5da8-e2c7-469a-a589-5f5f265f51d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:57.952001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e5da8-e2c7-469a-a589-5f5f265f51d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:57.952284 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e5da8-e2c7-469a-a589-5f5f265f51d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:57.952559 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e5da8-e2c7-469a-a589-5f5f265f51d7 admin admin] PATH_INFO: `/v3/projects/7420f594761449f1b59fc82dfea048b6/users/e2d38daf67f84ae09394628b2a2de74c/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:57.955139 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e5da8-e2c7-469a-a589-5f5f265f51d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7420f594761449f1b59fc82dfea048b6, user_id=e2d38daf67f84ae09394628b2a2de74c, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:57.957894 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e5da8-e2c7-469a-a589-5f5f265f51d7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:57.969963 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1130/2287] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:39:57 2022] PUT /identity/v3/projects/7420f594761449f1b59fc82dfea048b6/users/e2d38daf67f84ae09394628b2a2de74c/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:57.982038 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da999f9-8844-473c-8d10-54db031281b7 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:57.983793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da999f9-8844-473c-8d10-54db031281b7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:57.983793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da999f9-8844-473c-8d10-54db031281b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:58.012271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-5da999f9-8844-473c-8d10-54db031281b7 None None] MFA Rules not processed for user `e2d38daf67f84ae09394628b2a2de74c`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:58.013534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5da999f9-8844-473c-8d10-54db031281b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:58.088234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1158/2288] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:58.100601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e5da8-e2c7-469a-a589-5f5f265f51d7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:58.102463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d94e5da8-e2c7-469a-a589-5f5f265f51d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:58.160401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e5da8-e2c7-469a-a589-5f5f265f51d7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:58.163995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02774d90-4edc-46bc-a99e-3540844bbf4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:58.164641 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02774d90-4edc-46bc-a99e-3540844bbf4b service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:58.164926 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02774d90-4edc-46bc-a99e-3540844bbf4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:58.165228 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02774d90-4edc-46bc-a99e-3540844bbf4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:58.174200 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02774d90-4edc-46bc-a99e-3540844bbf4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:58.175820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02774d90-4edc-46bc-a99e-3540844bbf4b service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:58.192121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1131/2289] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:58.370922 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da999f9-8844-473c-8d10-54db031281b7 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:58.381402 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da999f9-8844-473c-8d10-54db031281b7 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:58.385421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487d24e-60f3-45c2-a4c2-1f822b46c797 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:58.386070 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487d24e-60f3-45c2-a4c2-1f822b46c797 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:58.386352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487d24e-60f3-45c2-a4c2-1f822b46c797 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:58.386623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487d24e-60f3-45c2-a4c2-1f822b46c797 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:58.446772 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487d24e-60f3-45c2-a4c2-1f822b46c797 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:58.448131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487d24e-60f3-45c2-a4c2-1f822b46c797 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:58.457687 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1159/2290] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:39:59.089078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02774d90-4edc-46bc-a99e-3540844bbf4b service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:59.186616 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02774d90-4edc-46bc-a99e-3540844bbf4b service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:39:59.193090 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c304f2f-c02d-4591-968b-e9f0877e23ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:39:59.194105 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c304f2f-c02d-4591-968b-e9f0877e23ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:59.194402 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c304f2f-c02d-4591-968b-e9f0877e23ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:59.194734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c304f2f-c02d-4591-968b-e9f0877e23ef admin admin] PATH_INFO: `/v3/users/2fab6a3a0aee40fbb0310c8bbc39c0a2` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:59.195938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c304f2f-c02d-4591-968b-e9f0877e23ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fab6a3a0aee40fbb0310c8bbc39c0a2)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:39:59.198031 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c304f2f-c02d-4591-968b-e9f0877e23ef admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:39:59.521594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c304f2f-c02d-4591-968b-e9f0877e23ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fab6a3a0aee40fbb0310c8bbc39c0a2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:59.522436 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c304f2f-c02d-4591-968b-e9f0877e23ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fab6a3a0aee40fbb0310c8bbc39c0a2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:59.529173 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:59.531155 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c304f2f-c02d-4591-968b-e9f0877e23ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fab6a3a0aee40fbb0310c8bbc39c0a2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:59.532064 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c304f2f-c02d-4591-968b-e9f0877e23ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fab6a3a0aee40fbb0310c8bbc39c0a2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:59.547494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c304f2f-c02d-4591-968b-e9f0877e23ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fab6a3a0aee40fbb0310c8bbc39c0a2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:39:59.626493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1132/2291] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:39:59 2022] DELETE /identity/v3/users/2fab6a3a0aee40fbb0310c8bbc39c0a2 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:39:59.631149 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e015c5-c3c2-409f-add4-9156a5edd94f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:39:59.631673 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e015c5-c3c2-409f-add4-9156a5edd94f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:39:59.631673 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e015c5-c3c2-409f-add4-9156a5edd94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:39:59.652587 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:39:59.730644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-75e015c5-c3c2-409f-add4-9156a5edd94f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:39:59.731576 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-75e015c5-c3c2-409f-add4-9156a5edd94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:39:59.866949 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1160/2292] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:39:59.878028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c304f2f-c02d-4591-968b-e9f0877e23ef admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:39:59.881649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9c304f2f-c02d-4591-968b-e9f0877e23ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:00.022552 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c304f2f-c02d-4591-968b-e9f0877e23ef admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:00.026538 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8df42-d633-45ed-b928-60e8cd51a877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:00.027218 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8df42-d633-45ed-b928-60e8cd51a877 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:00.027518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8df42-d633-45ed-b928-60e8cd51a877 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:00.027775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8df42-d633-45ed-b928-60e8cd51a877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:00.038434 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb8df42-d633-45ed-b928-60e8cd51a877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:00.039995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb8df42-d633-45ed-b928-60e8cd51a877 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:00.050480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1133/2293] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:39:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:40:00.517356 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e015c5-c3c2-409f-add4-9156a5edd94f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:00.522572 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-75e015c5-c3c2-409f-add4-9156a5edd94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:00.537201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e015c5-c3c2-409f-add4-9156a5edd94f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:00.542190 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01d6bf-434f-4d30-be83-aa5fc8b68234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:00.543508 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01d6bf-434f-4d30-be83-aa5fc8b68234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:00.543790 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01d6bf-434f-4d30-be83-aa5fc8b68234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:00.544318 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01d6bf-434f-4d30-be83-aa5fc8b68234 admin admin] PATH_INFO: `/v3/projects/110e8b0206ec41b4a6bd8f044b3f44e6` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:00.545735 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d01d6bf-434f-4d30-be83-aa5fc8b68234 admin admin] RBAC: Authorizing `identity:delete_project(project_id=110e8b0206ec41b4a6bd8f044b3f44e6)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:00.547651 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d01d6bf-434f-4d30-be83-aa5fc8b68234 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:00.672490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0d01d6bf-434f-4d30-be83-aa5fc8b68234 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '110e8b0206ec41b4a6bd8f044b3f44e6', 'request_id': 'req-0d01d6bf-434f-4d30-be83-aa5fc8b68234'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:00.678906 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0d01d6bf-434f-4d30-be83-aa5fc8b68234 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '110e8b0206ec41b4a6bd8f044b3f44e6', 'request_id': 'req-0d01d6bf-434f-4d30-be83-aa5fc8b68234'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:00.785604 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0d01d6bf-434f-4d30-be83-aa5fc8b68234 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '110e8b0206ec41b4a6bd8f044b3f44e6', 'request_id': 'req-0d01d6bf-434f-4d30-be83-aa5fc8b68234'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:00.800275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0d01d6bf-434f-4d30-be83-aa5fc8b68234 admin admin] The token cache is being invalidate because project 110e8b0206ec41b4a6bd8f044b3f44e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:40:00.800578 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0d01d6bf-434f-4d30-be83-aa5fc8b68234 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:00.811055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0d01d6bf-434f-4d30-be83-aa5fc8b68234 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:00.811055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1161/2294] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:40:00 2022] DELETE /identity/v3/projects/110e8b0206ec41b4a6bd8f044b3f44e6 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:40:00.811282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8df42-d633-45ed-b928-60e8cd51a877 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:00.820715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8df42-d633-45ed-b928-60e8cd51a877 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:00.902038 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae83b2-176b-4109-a3c2-5770d2a7a8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:00.903496 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae83b2-176b-4109-a3c2-5770d2a7a8b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:00.903937 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae83b2-176b-4109-a3c2-5770d2a7a8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:00.904360 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae83b2-176b-4109-a3c2-5770d2a7a8b4 admin admin] PATH_INFO: `/v3/users/e2d38daf67f84ae09394628b2a2de74c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:00.906151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae83b2-176b-4109-a3c2-5770d2a7a8b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2d38daf67f84ae09394628b2a2de74c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:00.908475 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae83b2-176b-4109-a3c2-5770d2a7a8b4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:01.155917 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:01.166196 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a3ae83b2-176b-4109-a3c2-5770d2a7a8b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2d38daf67f84ae09394628b2a2de74c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:01.166885 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a3ae83b2-176b-4109-a3c2-5770d2a7a8b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2d38daf67f84ae09394628b2a2de74c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:01.177104 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a3ae83b2-176b-4109-a3c2-5770d2a7a8b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2d38daf67f84ae09394628b2a2de74c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:01.178567 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a3ae83b2-176b-4109-a3c2-5770d2a7a8b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2d38daf67f84ae09394628b2a2de74c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:01.194719 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a3ae83b2-176b-4109-a3c2-5770d2a7a8b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2d38daf67f84ae09394628b2a2de74c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:01.293855 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1134/2295] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:40:00 2022] DELETE /identity/v3/users/e2d38daf67f84ae09394628b2a2de74c => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:40:01.856771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01d6bf-434f-4d30-be83-aa5fc8b68234 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:01.858960 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0d01d6bf-434f-4d30-be83-aa5fc8b68234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:01.876176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01d6bf-434f-4d30-be83-aa5fc8b68234 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:01.918126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:01.950328 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f67b7b-e952-49e7-8464-ec8ebcb9aca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:01.951191 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f67b7b-e952-49e7-8464-ec8ebcb9aca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:01.951471 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f67b7b-e952-49e7-8464-ec8ebcb9aca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:01.951765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f67b7b-e952-49e7-8464-ec8ebcb9aca4 admin admin] PATH_INFO: `/v3/projects/7420f594761449f1b59fc82dfea048b6` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:01.952940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f67b7b-e952-49e7-8464-ec8ebcb9aca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7420f594761449f1b59fc82dfea048b6)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:01.955031 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f67b7b-e952-49e7-8464-ec8ebcb9aca4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:02.082437 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c4f67b7b-e952-49e7-8464-ec8ebcb9aca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7420f594761449f1b59fc82dfea048b6', 'request_id': 'req-c4f67b7b-e952-49e7-8464-ec8ebcb9aca4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:02.089828 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c4f67b7b-e952-49e7-8464-ec8ebcb9aca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7420f594761449f1b59fc82dfea048b6', 'request_id': 'req-c4f67b7b-e952-49e7-8464-ec8ebcb9aca4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:02.315892 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c4f67b7b-e952-49e7-8464-ec8ebcb9aca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7420f594761449f1b59fc82dfea048b6', 'request_id': 'req-c4f67b7b-e952-49e7-8464-ec8ebcb9aca4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:02.328409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c4f67b7b-e952-49e7-8464-ec8ebcb9aca4 admin admin] The token cache is being invalidate because project 7420f594761449f1b59fc82dfea048b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:40:02.328694 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c4f67b7b-e952-49e7-8464-ec8ebcb9aca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:02.329350 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c4f67b7b-e952-49e7-8464-ec8ebcb9aca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:02.331028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1162/2296] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:40:01 2022] DELETE /identity/v3/projects/7420f594761449f1b59fc82dfea048b6 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:40:02.336174 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f097ad1-be49-43e6-a7e3-457893c6a21f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:02.336717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f097ad1-be49-43e6-a7e3-457893c6a21f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:02.336810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f097ad1-be49-43e6-a7e3-457893c6a21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:02.377646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0f097ad1-be49-43e6-a7e3-457893c6a21f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:02.378950 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0f097ad1-be49-43e6-a7e3-457893c6a21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:02.451292 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1135/2297] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:02.459088 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269603bc-b9ec-4d96-bd87-ba8d7b63cd37 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:02.459371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269603bc-b9ec-4d96-bd87-ba8d7b63cd37 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:02.459651 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269603bc-b9ec-4d96-bd87-ba8d7b63cd37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:02.478598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-269603bc-b9ec-4d96-bd87-ba8d7b63cd37 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:02.479799 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-269603bc-b9ec-4d96-bd87-ba8d7b63cd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:02.487628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1163/2298] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:02.493303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f097ad1-be49-43e6-a7e3-457893c6a21f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:02.503226 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f097ad1-be49-43e6-a7e3-457893c6a21f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:02.508487 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac4813-ceb7-477e-8995-8a99ca5b37f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:02.509197 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac4813-ceb7-477e-8995-8a99ca5b37f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:02.509486 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac4813-ceb7-477e-8995-8a99ca5b37f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:02.509770 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac4813-ceb7-477e-8995-8a99ca5b37f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:02.510508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac4813-ceb7-477e-8995-8a99ca5b37f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:40:02.510818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac4813-ceb7-477e-8995-8a99ca5b37f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:02.512427 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac4813-ceb7-477e-8995-8a99ca5b37f0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:02.524982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1136/2299] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:40:02.529439 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269603bc-b9ec-4d96-bd87-ba8d7b63cd37 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:02.539658 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269603bc-b9ec-4d96-bd87-ba8d7b63cd37 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:02.544367 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8140e8e-2786-4fe8-a383-f40837c64aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:02.545047 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8140e8e-2786-4fe8-a383-f40837c64aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:02.545332 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8140e8e-2786-4fe8-a383-f40837c64aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:02.545635 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8140e8e-2786-4fe8-a383-f40837c64aa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:02.546330 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8140e8e-2786-4fe8-a383-f40837c64aa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:02.548405 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8140e8e-2786-4fe8-a383-f40837c64aa3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:02.690576 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1164/2300] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:40:02 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:40:02.699674 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac4813-ceb7-477e-8995-8a99ca5b37f0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:02.710765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac4813-ceb7-477e-8995-8a99ca5b37f0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:02.775149 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dbabe8-e414-46c5-bbf0-40288a4f5ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:02.775865 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbabe8-e414-46c5-bbf0-40288a4f5ebd admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:02.776149 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbabe8-e414-46c5-bbf0-40288a4f5ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:02.776427 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbabe8-e414-46c5-bbf0-40288a4f5ebd admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:02.777166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dbabe8-e414-46c5-bbf0-40288a4f5ebd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:02.779124 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dbabe8-e414-46c5-bbf0-40288a4f5ebd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:02.879851 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1137/2301] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:40:02 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:40:02.881404 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8140e8e-2786-4fe8-a383-f40837c64aa3 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:02.899230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8140e8e-2786-4fe8-a383-f40837c64aa3 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:02.899576 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa92fc3-f009-4182-8c5c-a38038c55e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:02.900276 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa92fc3-f009-4182-8c5c-a38038c55e5a admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:02.900552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa92fc3-f009-4182-8c5c-a38038c55e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:02.900831 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa92fc3-f009-4182-8c5c-a38038c55e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:02.901759 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa92fc3-f009-4182-8c5c-a38038c55e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:02.903551 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa92fc3-f009-4182-8c5c-a38038c55e5a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:02.917405 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1165/2302] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:40:02.921507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dbabe8-e414-46c5-bbf0-40288a4f5ebd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:02.931998 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dbabe8-e414-46c5-bbf0-40288a4f5ebd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:02.936742 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e49f60-f8e8-4ecd-b792-281f1b5fb66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:02.937565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e49f60-f8e8-4ecd-b792-281f1b5fb66f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:02.937850 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e49f60-f8e8-4ecd-b792-281f1b5fb66f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:02.938130 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e49f60-f8e8-4ecd-b792-281f1b5fb66f admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:02.938865 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e49f60-f8e8-4ecd-b792-281f1b5fb66f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:02.940596 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e49f60-f8e8-4ecd-b792-281f1b5fb66f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:02.954544 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1138/2303] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:40:02.957474 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa92fc3-f009-4182-8c5c-a38038c55e5a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:02.968768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa92fc3-f009-4182-8c5c-a38038c55e5a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:02.974614 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f0ab5-775f-4115-aade-4ef87da8905a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:02.974979 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f0ab5-775f-4115-aade-4ef87da8905a admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:02.975312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f0ab5-775f-4115-aade-4ef87da8905a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:02.977438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f0ab5-775f-4115-aade-4ef87da8905a admin admin] PATH_INFO: `/v3/projects/eb6b987ebd1b48d19f31fc490d2e141f/users/7a28f31f320745649f8a2459caf20007/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:02.991631 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f0ab5-775f-4115-aade-4ef87da8905a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb6b987ebd1b48d19f31fc490d2e141f, user_id=7a28f31f320745649f8a2459caf20007, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:02.994346 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f0ab5-775f-4115-aade-4ef87da8905a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:03.082356 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1166/2304] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:40:02 2022] PUT /identity/v3/projects/eb6b987ebd1b48d19f31fc490d2e141f/users/7a28f31f320745649f8a2459caf20007/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:40:03.093287 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a67ee7-4df3-4b25-a964-d3a30f10d176 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:03.093710 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a67ee7-4df3-4b25-a964-d3a30f10d176 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:03.094083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a67ee7-4df3-4b25-a964-d3a30f10d176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:03.103912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:03.125240 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-04a67ee7-4df3-4b25-a964-d3a30f10d176 None None] MFA Rules not processed for user `7a28f31f320745649f8a2459caf20007`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:03.126548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-04a67ee7-4df3-4b25-a964-d3a30f10d176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:03.198894 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1139/2305] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:03.210435 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f0ab5-775f-4115-aade-4ef87da8905a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:03.212107 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e50f0ab5-775f-4115-aade-4ef87da8905a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:03.276774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f0ab5-775f-4115-aade-4ef87da8905a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:03.280889 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11e566-3b89-404b-afb6-2fb3212e572f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:03.281743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11e566-3b89-404b-afb6-2fb3212e572f service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:03.282187 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11e566-3b89-404b-afb6-2fb3212e572f service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:03.282512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11e566-3b89-404b-afb6-2fb3212e572f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:03.294594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11e566-3b89-404b-afb6-2fb3212e572f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:03.296133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11e566-3b89-404b-afb6-2fb3212e572f service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:03.306341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1167/2306] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:40:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:40:13.368121 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:13.596125 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=30 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:24.357659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4e5b3-f7ef-48e2-b538-de662c200649 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:24.357659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4e5b3-f7ef-48e2-b538-de662c200649 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:24.357659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4e5b3-f7ef-48e2-b538-de662c200649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:24.370663 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d5b4e5b3-f7ef-48e2-b538-de662c200649 None None] MFA Rules not processed for user `489306774e664047a2a5629048455686`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:24.371909 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d5b4e5b3-f7ef-48e2-b538-de662c200649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:24.438125 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1140/2307] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:30.797960 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575721d-248f-4313-a9a4-deb16c568a7b None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:30.798642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575721d-248f-4313-a9a4-deb16c568a7b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:30.798642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575721d-248f-4313-a9a4-deb16c568a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:30.822102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e575721d-248f-4313-a9a4-deb16c568a7b None None] MFA Rules not processed for user `055ec8a54bdf400fac0a46b0e5497f1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:30.823334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e575721d-248f-4313-a9a4-deb16c568a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:30.897514 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1168/2308] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:32.791978 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cdaa4-cf36-417f-b3e1-c328a21f4265 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:32.792927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cdaa4-cf36-417f-b3e1-c328a21f4265 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:32.793384 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cdaa4-cf36-417f-b3e1-c328a21f4265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:32.845657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-498cdaa4-cf36-417f-b3e1-c328a21f4265 None None] MFA Rules not processed for user `bbf0ce396885472bb5d225aa009043f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:32.846966 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-498cdaa4-cf36-417f-b3e1-c328a21f4265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:32.928550 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1141/2309] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jun 7 08:40:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:33.008894 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575721d-248f-4313-a9a4-deb16c568a7b None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:33.011296 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e575721d-248f-4313-a9a4-deb16c568a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:33.113404 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575721d-248f-4313-a9a4-deb16c568a7b None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:33.120662 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb883b2-3a58-4164-90e9-582b0f924bfe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334dc47ae2e643ccb44d25c6b5906f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:33.123237 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb883b2-3a58-4164-90e9-582b0f924bfe service swift] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:33.123685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb883b2-3a58-4164-90e9-582b0f924bfe service swift] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:33.124109 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb883b2-3a58-4164-90e9-582b0f924bfe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:33.139104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb883b2-3a58-4164-90e9-582b0f924bfe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:33.139104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb883b2-3a58-4164-90e9-582b0f924bfe service swift] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:33.152578 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1169/2310] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jun 7 08:40:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:40:35.805349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cdaa4-cf36-417f-b3e1-c328a21f4265 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:35.806839 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-498cdaa4-cf36-417f-b3e1-c328a21f4265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:35.921817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cdaa4-cf36-417f-b3e1-c328a21f4265 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:35.927117 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3fb38c-53d4-4fde-bb86-7843893544c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:35.928371 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fb38c-53d4-4fde-bb86-7843893544c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:35.929552 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fb38c-53d4-4fde-bb86-7843893544c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:35.930012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fb38c-53d4-4fde-bb86-7843893544c2 admin admin] PATH_INFO: `/v3/users/801238c8c0c74d69bea97da67faf7141` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:35.931343 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3fb38c-53d4-4fde-bb86-7843893544c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=801238c8c0c74d69bea97da67faf7141)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:35.934432 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3fb38c-53d4-4fde-bb86-7843893544c2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:36.204316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:36.218993 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7f3fb38c-53d4-4fde-bb86-7843893544c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '801238c8c0c74d69bea97da67faf7141'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:36.224092 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7f3fb38c-53d4-4fde-bb86-7843893544c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '801238c8c0c74d69bea97da67faf7141'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:36.244749 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7f3fb38c-53d4-4fde-bb86-7843893544c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '801238c8c0c74d69bea97da67faf7141'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:36.246116 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7f3fb38c-53d4-4fde-bb86-7843893544c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '801238c8c0c74d69bea97da67faf7141'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:36.273898 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7f3fb38c-53d4-4fde-bb86-7843893544c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '801238c8c0c74d69bea97da67faf7141'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:36.421813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1142/2311] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:40:35 2022] DELETE /identity/v3/users/801238c8c0c74d69bea97da67faf7141 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:40:36.438946 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67323d-755d-4981-9b1f-a72f01e6162f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:36.439766 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67323d-755d-4981-9b1f-a72f01e6162f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:36.440335 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67323d-755d-4981-9b1f-a72f01e6162f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:36.465688 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-4c67323d-755d-4981-9b1f-a72f01e6162f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:36.467090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4c67323d-755d-4981-9b1f-a72f01e6162f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:36.552502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1170/2312] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:36.563536 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3fb38c-53d4-4fde-bb86-7843893544c2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:36.576894 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7f3fb38c-53d4-4fde-bb86-7843893544c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:36.652768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3fb38c-53d4-4fde-bb86-7843893544c2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:36.656375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c31b60-6e6f-42a2-b2fd-19fe505dc79b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:36.657039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c31b60-6e6f-42a2-b2fd-19fe505dc79b service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:36.657321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c31b60-6e6f-42a2-b2fd-19fe505dc79b service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:36.657592 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c31b60-6e6f-42a2-b2fd-19fe505dc79b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:36.667066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c31b60-6e6f-42a2-b2fd-19fe505dc79b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:36.668505 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c31b60-6e6f-42a2-b2fd-19fe505dc79b service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:36.696768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1143/2313] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:40:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:40:37.420583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c67323d-755d-4981-9b1f-a72f01e6162f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:37.423917 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4c67323d-755d-4981-9b1f-a72f01e6162f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:37.444108 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c67323d-755d-4981-9b1f-a72f01e6162f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:37.464522 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed058a6-d4f8-4f28-91d5-836afd6696c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:37.465446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed058a6-d4f8-4f28-91d5-836afd6696c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:37.465726 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed058a6-d4f8-4f28-91d5-836afd6696c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:37.466040 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed058a6-d4f8-4f28-91d5-836afd6696c5 admin admin] PATH_INFO: `/v3/projects/39536aa9e96c46eb8e6e62a74d5db850` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:37.470906 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed058a6-d4f8-4f28-91d5-836afd6696c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39536aa9e96c46eb8e6e62a74d5db850)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:37.477428 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed058a6-d4f8-4f28-91d5-836afd6696c5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:37.686006 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:37.723210 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bed058a6-d4f8-4f28-91d5-836afd6696c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39536aa9e96c46eb8e6e62a74d5db850', 'request_id': 'req-bed058a6-d4f8-4f28-91d5-836afd6696c5'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:37.737879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bed058a6-d4f8-4f28-91d5-836afd6696c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39536aa9e96c46eb8e6e62a74d5db850', 'request_id': 'req-bed058a6-d4f8-4f28-91d5-836afd6696c5'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:37.823792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bed058a6-d4f8-4f28-91d5-836afd6696c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39536aa9e96c46eb8e6e62a74d5db850', 'request_id': 'req-bed058a6-d4f8-4f28-91d5-836afd6696c5'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:37.842593 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bed058a6-d4f8-4f28-91d5-836afd6696c5 admin admin] The token cache is being invalidate because project 39536aa9e96c46eb8e6e62a74d5db850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:40:37.843138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bed058a6-d4f8-4f28-91d5-836afd6696c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:37.844076 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bed058a6-d4f8-4f28-91d5-836afd6696c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:37.846594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1171/2314] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:40:37 2022] DELETE /identity/v3/projects/39536aa9e96c46eb8e6e62a74d5db850 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:40:39.587173 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c31b60-6e6f-42a2-b2fd-19fe505dc79b service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:39.588968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c2c31b60-6e6f-42a2-b2fd-19fe505dc79b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:39.618315 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c31b60-6e6f-42a2-b2fd-19fe505dc79b service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:39.707349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26934368-8a75-4c52-a0d7-f34f07bde2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:39.708769 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26934368-8a75-4c52-a0d7-f34f07bde2d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:39.708769 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26934368-8a75-4c52-a0d7-f34f07bde2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:39.709379 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26934368-8a75-4c52-a0d7-f34f07bde2d8 admin admin] PATH_INFO: `/v3/users/7a28f31f320745649f8a2459caf20007` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:39.710853 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26934368-8a75-4c52-a0d7-f34f07bde2d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a28f31f320745649f8a2459caf20007)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:39.713230 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26934368-8a75-4c52-a0d7-f34f07bde2d8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:39.943752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-26934368-8a75-4c52-a0d7-f34f07bde2d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a28f31f320745649f8a2459caf20007'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:39.945434 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-26934368-8a75-4c52-a0d7-f34f07bde2d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a28f31f320745649f8a2459caf20007'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:39.956513 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-26934368-8a75-4c52-a0d7-f34f07bde2d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a28f31f320745649f8a2459caf20007'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:39.956675 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-26934368-8a75-4c52-a0d7-f34f07bde2d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a28f31f320745649f8a2459caf20007'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:39.959657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:39.976803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-26934368-8a75-4c52-a0d7-f34f07bde2d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a28f31f320745649f8a2459caf20007'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:40.065531 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1144/2315] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:40:39 2022] DELETE /identity/v3/users/7a28f31f320745649f8a2459caf20007 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:40:40.070929 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1fd28-cf6b-47d5-b126-26ab7c321dd6 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:40.071830 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1fd28-cf6b-47d5-b126-26ab7c321dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:40.072354 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1fd28-cf6b-47d5-b126-26ab7c321dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:40.095198 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7ff1fd28-cf6b-47d5-b126-26ab7c321dd6 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:40.097292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7ff1fd28-cf6b-47d5-b126-26ab7c321dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:40.192582 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26934368-8a75-4c52-a0d7-f34f07bde2d8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:40.193188 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1172/2316] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:40.195284 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-26934368-8a75-4c52-a0d7-f34f07bde2d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:40.328491 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26934368-8a75-4c52-a0d7-f34f07bde2d8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:40.337020 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe24a48-0da1-4e5e-a505-e69c73fc173e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:40.340449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe24a48-0da1-4e5e-a505-e69c73fc173e service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:40.341019 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe24a48-0da1-4e5e-a505-e69c73fc173e service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:40.341500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe24a48-0da1-4e5e-a505-e69c73fc173e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:40.361281 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe24a48-0da1-4e5e-a505-e69c73fc173e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:40.362842 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe24a48-0da1-4e5e-a505-e69c73fc173e service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:40.374362 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1145/2317] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:40:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:40:40.938030 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1fd28-cf6b-47d5-b126-26ab7c321dd6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:40.939583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7ff1fd28-cf6b-47d5-b126-26ab7c321dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:40.953127 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1fd28-cf6b-47d5-b126-26ab7c321dd6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:40.967812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4430de-2182-45b2-ad14-24b486192154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:40.968834 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4430de-2182-45b2-ad14-24b486192154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:40.969268 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4430de-2182-45b2-ad14-24b486192154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:40.969792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4430de-2182-45b2-ad14-24b486192154 admin admin] PATH_INFO: `/v3/projects/eb6b987ebd1b48d19f31fc490d2e141f` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:40.971327 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4430de-2182-45b2-ad14-24b486192154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb6b987ebd1b48d19f31fc490d2e141f)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:40.973689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4430de-2182-45b2-ad14-24b486192154 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:41.188558 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8e4430de-2182-45b2-ad14-24b486192154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb6b987ebd1b48d19f31fc490d2e141f', 'request_id': 'req-8e4430de-2182-45b2-ad14-24b486192154'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:41.200457 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8e4430de-2182-45b2-ad14-24b486192154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb6b987ebd1b48d19f31fc490d2e141f', 'request_id': 'req-8e4430de-2182-45b2-ad14-24b486192154'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:41.750738 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8e4430de-2182-45b2-ad14-24b486192154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb6b987ebd1b48d19f31fc490d2e141f', 'request_id': 'req-8e4430de-2182-45b2-ad14-24b486192154'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:41.768639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8e4430de-2182-45b2-ad14-24b486192154 admin admin] The token cache is being invalidate because project eb6b987ebd1b48d19f31fc490d2e141f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:40:41.769105 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8e4430de-2182-45b2-ad14-24b486192154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:41.771032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8e4430de-2182-45b2-ad14-24b486192154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:41.778745 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1173/2318] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:40:40 2022] DELETE /identity/v3/projects/eb6b987ebd1b48d19f31fc490d2e141f => generated 0 bytes in 837 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:40:41.782867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51b93d-9102-40ab-8cf5-8e74612c35a1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:41.782867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51b93d-9102-40ab-8cf5-8e74612c35a1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:41.782867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51b93d-9102-40ab-8cf5-8e74612c35a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:41.815840 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1e51b93d-9102-40ab-8cf5-8e74612c35a1 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:41.817792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1e51b93d-9102-40ab-8cf5-8e74612c35a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:42.003045 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c413ac-e25d-4cd9-ad88-36458d31b5be None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:42.005102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c413ac-e25d-4cd9-ad88-36458d31b5be None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:42.005102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c413ac-e25d-4cd9-ad88-36458d31b5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:42.005610 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1146/2319] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:42.028032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-86c413ac-e25d-4cd9-ad88-36458d31b5be None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:42.032196 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-86c413ac-e25d-4cd9-ad88-36458d31b5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:42.045043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51b93d-9102-40ab-8cf5-8e74612c35a1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:42.057446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1174/2320] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:40:42.062677 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51b93d-9102-40ab-8cf5-8e74612c35a1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:42.068263 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd12e4-e692-46d7-8545-d5e5318ad5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:42.069579 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd12e4-e692-46d7-8545-d5e5318ad5b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:42.069860 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd12e4-e692-46d7-8545-d5e5318ad5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:42.070338 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd12e4-e692-46d7-8545-d5e5318ad5b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:42.071222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd12e4-e692-46d7-8545-d5e5318ad5b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:40:42.071610 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd12e4-e692-46d7-8545-d5e5318ad5b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:42.076480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd12e4-e692-46d7-8545-d5e5318ad5b3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:42.092441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1147/2321] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:40:42.109785 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c413ac-e25d-4cd9-ad88-36458d31b5be None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:42.120846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c413ac-e25d-4cd9-ad88-36458d31b5be None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:42.125907 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a944492-5f8a-4bb2-aed2-6ae8aa4ca505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:42.129433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a944492-5f8a-4bb2-aed2-6ae8aa4ca505 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:42.129433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a944492-5f8a-4bb2-aed2-6ae8aa4ca505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:42.129433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a944492-5f8a-4bb2-aed2-6ae8aa4ca505 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:42.129433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a944492-5f8a-4bb2-aed2-6ae8aa4ca505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:42.131070 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a944492-5f8a-4bb2-aed2-6ae8aa4ca505 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:42.275595 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1175/2322] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:40:42 2022] POST /identity/v3/projects => generated 381 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:40:42.280443 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd12e4-e692-46d7-8545-d5e5318ad5b3 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:42.292965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd12e4-e692-46d7-8545-d5e5318ad5b3 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:42.353509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:42.360486 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce49290-7c7f-47be-8731-5cc576c72dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:42.361182 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce49290-7c7f-47be-8731-5cc576c72dfa admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:42.361465 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce49290-7c7f-47be-8731-5cc576c72dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:42.361740 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce49290-7c7f-47be-8731-5cc576c72dfa admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:42.362436 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce49290-7c7f-47be-8731-5cc576c72dfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:42.364393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce49290-7c7f-47be-8731-5cc576c72dfa admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:42.424783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1148/2323] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:40:42 2022] POST /identity/v3/users => generated 291 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:40:42.430842 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a944492-5f8a-4bb2-aed2-6ae8aa4ca505 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:42.443002 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a944492-5f8a-4bb2-aed2-6ae8aa4ca505 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:42.453529 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b83731-21ad-4070-8380-6e1d2817779c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:42.454226 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b83731-21ad-4070-8380-6e1d2817779c admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:42.454518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b83731-21ad-4070-8380-6e1d2817779c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:42.457260 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b83731-21ad-4070-8380-6e1d2817779c admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:42.459036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b83731-21ad-4070-8380-6e1d2817779c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:42.461954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b83731-21ad-4070-8380-6e1d2817779c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:42.479826 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1176/2324] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:40:42.487042 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce49290-7c7f-47be-8731-5cc576c72dfa admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:42.503408 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce49290-7c7f-47be-8731-5cc576c72dfa admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:42.517726 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce9986-ecb5-40c7-9e7a-15c42218e189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:42.518430 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce9986-ecb5-40c7-9e7a-15c42218e189 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:42.518713 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce9986-ecb5-40c7-9e7a-15c42218e189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:42.518987 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce9986-ecb5-40c7-9e7a-15c42218e189 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:42.519794 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce9986-ecb5-40c7-9e7a-15c42218e189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:42.521501 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce9986-ecb5-40c7-9e7a-15c42218e189 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:42.536583 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1149/2325] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:40:42.541374 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b83731-21ad-4070-8380-6e1d2817779c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:42.547839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:42.554315 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b83731-21ad-4070-8380-6e1d2817779c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:42.561562 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b2393-bfe7-4e25-9b6e-23b54f401ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:42.562906 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b2393-bfe7-4e25-9b6e-23b54f401ec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:42.563048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b2393-bfe7-4e25-9b6e-23b54f401ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:42.563336 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b2393-bfe7-4e25-9b6e-23b54f401ec5 admin admin] PATH_INFO: `/v3/projects/083e79f9fe2a412c95b456e5bc6ff93e/users/3f9edf9cac334601badb5df2c77cac42/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:42.599019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b2393-bfe7-4e25-9b6e-23b54f401ec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=083e79f9fe2a412c95b456e5bc6ff93e, user_id=3f9edf9cac334601badb5df2c77cac42, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:42.602950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b2393-bfe7-4e25-9b6e-23b54f401ec5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:42.681110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1177/2326] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:40:42 2022] PUT /identity/v3/projects/083e79f9fe2a412c95b456e5bc6ff93e/users/3f9edf9cac334601badb5df2c77cac42/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:40:42.694336 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c24cd-c843-42c5-8c84-49f9e333fea8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:42.695196 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c24cd-c843-42c5-8c84-49f9e333fea8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:42.695918 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c24cd-c843-42c5-8c84-49f9e333fea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:42.736722 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a46c24cd-c843-42c5-8c84-49f9e333fea8 None None] MFA Rules not processed for user `3f9edf9cac334601badb5df2c77cac42`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:42.738154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a46c24cd-c843-42c5-8c84-49f9e333fea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:42.856051 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1150/2327] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:42.863381 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b2393-bfe7-4e25-9b6e-23b54f401ec5 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:42.971774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b2393-bfe7-4e25-9b6e-23b54f401ec5 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:42.977543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33df0a-4ab2-47a5-a67d-bfc1e1d764de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:42.978153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33df0a-4ab2-47a5-a67d-bfc1e1d764de admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:42.978502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33df0a-4ab2-47a5-a67d-bfc1e1d764de admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:42.978848 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33df0a-4ab2-47a5-a67d-bfc1e1d764de admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:42.981146 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d33df0a-4ab2-47a5-a67d-bfc1e1d764de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:42.984942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d33df0a-4ab2-47a5-a67d-bfc1e1d764de admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:43.089685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1178/2328] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:40:42 2022] POST /identity/v3/projects => generated 385 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:40:43.094357 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c24cd-c843-42c5-8c84-49f9e333fea8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:43.109321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c24cd-c843-42c5-8c84-49f9e333fea8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:43.399197 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426adb34-b0ee-4afb-8e02-39f47a8aadab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:43.399878 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426adb34-b0ee-4afb-8e02-39f47a8aadab admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:43.400157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426adb34-b0ee-4afb-8e02-39f47a8aadab admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:43.400455 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426adb34-b0ee-4afb-8e02-39f47a8aadab admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:43.401149 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426adb34-b0ee-4afb-8e02-39f47a8aadab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:43.403068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426adb34-b0ee-4afb-8e02-39f47a8aadab admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:43.535481 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1151/2329] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:40:43 2022] POST /identity/v3/users => generated 293 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:40:43.540653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33df0a-4ab2-47a5-a67d-bfc1e1d764de admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:43.556249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33df0a-4ab2-47a5-a67d-bfc1e1d764de admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:43.561729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49eecd-4acf-4827-bf17-9a3438008afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:43.562426 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49eecd-4acf-4827-bf17-9a3438008afa admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:43.562708 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49eecd-4acf-4827-bf17-9a3438008afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:43.562985 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49eecd-4acf-4827-bf17-9a3438008afa admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:43.563726 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49eecd-4acf-4827-bf17-9a3438008afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:43.565419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49eecd-4acf-4827-bf17-9a3438008afa admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:43.601207 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1179/2330] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:40:43.604324 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426adb34-b0ee-4afb-8e02-39f47a8aadab admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:43.619874 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426adb34-b0ee-4afb-8e02-39f47a8aadab admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:43.627968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545236b-b9a2-4904-9496-1c11900ba953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:43.631190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545236b-b9a2-4904-9496-1c11900ba953 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:43.631505 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545236b-b9a2-4904-9496-1c11900ba953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:43.631791 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545236b-b9a2-4904-9496-1c11900ba953 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:43.633483 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545236b-b9a2-4904-9496-1c11900ba953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:43.639166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545236b-b9a2-4904-9496-1c11900ba953 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:43.660459 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1152/2331] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:40:43.672317 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49eecd-4acf-4827-bf17-9a3438008afa admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:43.692852 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49eecd-4acf-4827-bf17-9a3438008afa admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:43.703252 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0a42d-c0a2-4018-963b-062e72f9ba4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:43.704770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0a42d-c0a2-4018-963b-062e72f9ba4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:43.705161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0a42d-c0a2-4018-963b-062e72f9ba4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:43.705520 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0a42d-c0a2-4018-963b-062e72f9ba4e admin admin] PATH_INFO: `/v3/projects/543d059d266a45e189115b68bb4effcb/users/1b199b01e3554bb38f747125a7250593/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:43.725292 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a0a42d-c0a2-4018-963b-062e72f9ba4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=543d059d266a45e189115b68bb4effcb, user_id=1b199b01e3554bb38f747125a7250593, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:43.728813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a0a42d-c0a2-4018-963b-062e72f9ba4e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:43.842844 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1180/2332] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:40:43 2022] PUT /identity/v3/projects/543d059d266a45e189115b68bb4effcb/users/1b199b01e3554bb38f747125a7250593/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:40:43.854543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56dbc57-42bf-4107-b5a8-47947e6bc785 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:43.854900 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56dbc57-42bf-4107-b5a8-47947e6bc785 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:43.858308 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56dbc57-42bf-4107-b5a8-47947e6bc785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:43.892506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b56dbc57-42bf-4107-b5a8-47947e6bc785 None None] MFA Rules not processed for user `1b199b01e3554bb38f747125a7250593`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:43.892506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b56dbc57-42bf-4107-b5a8-47947e6bc785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:43.956904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:43.995352 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1153/2333] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:40:43.999856 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0a42d-c0a2-4018-963b-062e72f9ba4e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:44.117597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0a42d-c0a2-4018-963b-062e72f9ba4e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:44.131248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d9314-a4c9-4b8f-8cc2-afb70daf100b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:44.131964 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d9314-a4c9-4b8f-8cc2-afb70daf100b admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:44.132254 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d9314-a4c9-4b8f-8cc2-afb70daf100b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:44.132533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d9314-a4c9-4b8f-8cc2-afb70daf100b admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:44.133309 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d9314-a4c9-4b8f-8cc2-afb70daf100b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:44.136049 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d9314-a4c9-4b8f-8cc2-afb70daf100b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:44.261286 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1181/2334] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:40:43 2022] POST /identity/v3/projects => generated 385 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:40:44.262853 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56dbc57-42bf-4107-b5a8-47947e6bc785 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:44.274973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56dbc57-42bf-4107-b5a8-47947e6bc785 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:44.357232 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef21d3d-bd07-4bef-9985-dab9f7c80be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:44.358379 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef21d3d-bd07-4bef-9985-dab9f7c80be7 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:44.358876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef21d3d-bd07-4bef-9985-dab9f7c80be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:44.359363 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef21d3d-bd07-4bef-9985-dab9f7c80be7 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:44.360262 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef21d3d-bd07-4bef-9985-dab9f7c80be7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:44.362460 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef21d3d-bd07-4bef-9985-dab9f7c80be7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:44.452299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1154/2335] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:40:44 2022] POST /identity/v3/users => generated 299 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:40:44.452563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d9314-a4c9-4b8f-8cc2-afb70daf100b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:44.466877 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d9314-a4c9-4b8f-8cc2-afb70daf100b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:44.473675 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bf83b-2101-4e0b-b1cf-a746b7ecf1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:44.474594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bf83b-2101-4e0b-b1cf-a746b7ecf1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:44.475980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bf83b-2101-4e0b-b1cf-a746b7ecf1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:44.475980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bf83b-2101-4e0b-b1cf-a746b7ecf1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:44.475980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bf83b-2101-4e0b-b1cf-a746b7ecf1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:44.478565 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bf83b-2101-4e0b-b1cf-a746b7ecf1d8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:44.497837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1182/2336] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:40:44.502000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef21d3d-bd07-4bef-9985-dab9f7c80be7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:44.517112 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef21d3d-bd07-4bef-9985-dab9f7c80be7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:44.528417 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b98d7b-8d66-4c10-b822-0a9489b4ed4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:44.530222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b98d7b-8d66-4c10-b822-0a9489b4ed4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:44.530721 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b98d7b-8d66-4c10-b822-0a9489b4ed4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:44.531195 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b98d7b-8d66-4c10-b822-0a9489b4ed4d admin admin] PATH_INFO: `/v3/projects/f6a021fad0b2429f9a98885fa262ad10/users/e60ab3bc7f0943a4a654268cdde6f0a6/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:44.557251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b98d7b-8d66-4c10-b822-0a9489b4ed4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6a021fad0b2429f9a98885fa262ad10, user_id=e60ab3bc7f0943a4a654268cdde6f0a6, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:44.561326 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b98d7b-8d66-4c10-b822-0a9489b4ed4d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:44.627956 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1155/2337] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:40:44 2022] PUT /identity/v3/projects/f6a021fad0b2429f9a98885fa262ad10/users/e60ab3bc7f0943a4a654268cdde6f0a6/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:40:44.632620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bf83b-2101-4e0b-b1cf-a746b7ecf1d8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:44.638823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:44.644520 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bf83b-2101-4e0b-b1cf-a746b7ecf1d8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:44.717221 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5d9e0-06f4-42b7-acbb-1e733a2a68ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:44.718158 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5d9e0-06f4-42b7-acbb-1e733a2a68ff admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:44.718443 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5d9e0-06f4-42b7-acbb-1e733a2a68ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:44.718748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5d9e0-06f4-42b7-acbb-1e733a2a68ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:44.719476 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5d9e0-06f4-42b7-acbb-1e733a2a68ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:44.721120 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5d9e0-06f4-42b7-acbb-1e733a2a68ff admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:44.734015 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1183/2338] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:40:44.738434 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b98d7b-8d66-4c10-b822-0a9489b4ed4d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:44.775377 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b98d7b-8d66-4c10-b822-0a9489b4ed4d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:44.786131 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd547bd-5f03-4358-bf0f-660c60e8a9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:44.787214 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd547bd-5f03-4358-bf0f-660c60e8a9f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:44.792425 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd547bd-5f03-4358-bf0f-660c60e8a9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:44.792425 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd547bd-5f03-4358-bf0f-660c60e8a9f8 admin admin] PATH_INFO: `/v3/projects/f6a021fad0b2429f9a98885fa262ad10/users/e60ab3bc7f0943a4a654268cdde6f0a6/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:44.792425 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd547bd-5f03-4358-bf0f-660c60e8a9f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6a021fad0b2429f9a98885fa262ad10, user_id=e60ab3bc7f0943a4a654268cdde6f0a6, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:44.794978 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd547bd-5f03-4358-bf0f-660c60e8a9f8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:44.808102 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1156/2339] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:40:44 2022] PUT /identity/v3/projects/f6a021fad0b2429f9a98885fa262ad10/users/e60ab3bc7f0943a4a654268cdde6f0a6/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:40:44.819012 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e092356-474c-4103-8d17-87a22f9a64e9 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:44.826496 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e092356-474c-4103-8d17-87a22f9a64e9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:44.826496 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e092356-474c-4103-8d17-87a22f9a64e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:44.859249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-2e092356-474c-4103-8d17-87a22f9a64e9 None None] MFA Rules not processed for user `e60ab3bc7f0943a4a654268cdde6f0a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:44.859993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2e092356-474c-4103-8d17-87a22f9a64e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:44.986603 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1184/2340] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:45.010269 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd547bd-5f03-4358-bf0f-660c60e8a9f8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:45.022315 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1dd547bd-5f03-4358-bf0f-660c60e8a9f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:45.269216 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd547bd-5f03-4358-bf0f-660c60e8a9f8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:45.305096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5c97d-54ec-4fda-8211-eebd9ca3f75e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:45.305096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5c97d-54ec-4fda-8211-eebd9ca3f75e service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:45.305096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5c97d-54ec-4fda-8211-eebd9ca3f75e service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:45.305096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5c97d-54ec-4fda-8211-eebd9ca3f75e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:45.344454 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:45.409039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5c97d-54ec-4fda-8211-eebd9ca3f75e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:45.410723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5c97d-54ec-4fda-8211-eebd9ca3f75e service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:45.421276 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1157/2341] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:40:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:40:47.869643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e092356-474c-4103-8d17-87a22f9a64e9 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:47.871476 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2e092356-474c-4103-8d17-87a22f9a64e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:47.884769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e092356-474c-4103-8d17-87a22f9a64e9 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:47.888708 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e936532-2b6f-4799-a440-1a29ea077c39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:47.889418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e936532-2b6f-4799-a440-1a29ea077c39 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:47.889702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e936532-2b6f-4799-a440-1a29ea077c39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:47.890011 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e936532-2b6f-4799-a440-1a29ea077c39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:47.964479 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e936532-2b6f-4799-a440-1a29ea077c39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:47.966006 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e936532-2b6f-4799-a440-1a29ea077c39 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:47.975919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1185/2342] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:40:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:40:49.492888 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5c97d-54ec-4fda-8211-eebd9ca3f75e service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:49.502417 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5c97d-54ec-4fda-8211-eebd9ca3f75e service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:49.506418 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fded8a-d4df-4f77-95c7-29aca8ce3bbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:49.507712 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fded8a-d4df-4f77-95c7-29aca8ce3bbb service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:49.508092 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fded8a-d4df-4f77-95c7-29aca8ce3bbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:49.508477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fded8a-d4df-4f77-95c7-29aca8ce3bbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:49.520075 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fded8a-d4df-4f77-95c7-29aca8ce3bbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:49.521826 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fded8a-d4df-4f77-95c7-29aca8ce3bbb service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:49.541484 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1158/2343] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:40:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:40:52.477490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e936532-2b6f-4799-a440-1a29ea077c39 service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:52.479006 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2e936532-2b6f-4799-a440-1a29ea077c39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:52.508330 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:52.560960 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e936532-2b6f-4799-a440-1a29ea077c39 service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:52.566166 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021e65e-b5c9-4a9d-8228-4efb498975f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:52.567517 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021e65e-b5c9-4a9d-8228-4efb498975f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:52.568533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021e65e-b5c9-4a9d-8228-4efb498975f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:52.569829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021e65e-b5c9-4a9d-8228-4efb498975f5 admin admin] PATH_INFO: `/v3/users/f9945de895484b66acf146b131043f40` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:52.573319 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021e65e-b5c9-4a9d-8228-4efb498975f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9945de895484b66acf146b131043f40)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:52.576456 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021e65e-b5c9-4a9d-8228-4efb498975f5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:52.892153 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2021e65e-b5c9-4a9d-8228-4efb498975f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9945de895484b66acf146b131043f40'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:52.892970 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2021e65e-b5c9-4a9d-8228-4efb498975f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9945de895484b66acf146b131043f40'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:52.905788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2021e65e-b5c9-4a9d-8228-4efb498975f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9945de895484b66acf146b131043f40'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:52.906520 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2021e65e-b5c9-4a9d-8228-4efb498975f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9945de895484b66acf146b131043f40'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:52.978214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2021e65e-b5c9-4a9d-8228-4efb498975f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9945de895484b66acf146b131043f40'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:53.003069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1186/2344] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:40:52 2022] DELETE /identity/v3/users/f9945de895484b66acf146b131043f40 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:40:53.008251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af8be5-99d4-43d8-95cd-b75a36b50b4c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:53.008773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af8be5-99d4-43d8-95cd-b75a36b50b4c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:53.009171 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af8be5-99d4-43d8-95cd-b75a36b50b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:53.030799 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-23af8be5-99d4-43d8-95cd-b75a36b50b4c None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:53.032115 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-23af8be5-99d4-43d8-95cd-b75a36b50b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:53.108621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1159/2345] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:53.117350 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021e65e-b5c9-4a9d-8228-4efb498975f5 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:53.118186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2021e65e-b5c9-4a9d-8228-4efb498975f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:53.189651 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021e65e-b5c9-4a9d-8228-4efb498975f5 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:53.193629 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417e1e0-827e-403e-9842-76ae8ed4ee32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:53.194517 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417e1e0-827e-403e-9842-76ae8ed4ee32 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:53.194898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417e1e0-827e-403e-9842-76ae8ed4ee32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:53.195245 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417e1e0-827e-403e-9842-76ae8ed4ee32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:53.205730 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417e1e0-827e-403e-9842-76ae8ed4ee32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:53.207451 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417e1e0-827e-403e-9842-76ae8ed4ee32 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:53.218182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1187/2346] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:40:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:40:53.690440 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af8be5-99d4-43d8-95cd-b75a36b50b4c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:53.692959 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-23af8be5-99d4-43d8-95cd-b75a36b50b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:53.713916 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af8be5-99d4-43d8-95cd-b75a36b50b4c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:53.729734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501eda15-7ac3-4326-aeef-475d04aaf2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:53.731187 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eda15-7ac3-4326-aeef-475d04aaf2bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:53.731478 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eda15-7ac3-4326-aeef-475d04aaf2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:53.731758 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eda15-7ac3-4326-aeef-475d04aaf2bf admin admin] PATH_INFO: `/v3/projects/53f3a1dc6bbf4cd382c7f507fed85049` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:53.733257 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501eda15-7ac3-4326-aeef-475d04aaf2bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=53f3a1dc6bbf4cd382c7f507fed85049)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:53.742823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501eda15-7ac3-4326-aeef-475d04aaf2bf admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:53.836568 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-501eda15-7ac3-4326-aeef-475d04aaf2bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53f3a1dc6bbf4cd382c7f507fed85049', 'request_id': 'req-501eda15-7ac3-4326-aeef-475d04aaf2bf'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:53.843644 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-501eda15-7ac3-4326-aeef-475d04aaf2bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53f3a1dc6bbf4cd382c7f507fed85049', 'request_id': 'req-501eda15-7ac3-4326-aeef-475d04aaf2bf'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:53.851098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-501eda15-7ac3-4326-aeef-475d04aaf2bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53f3a1dc6bbf4cd382c7f507fed85049', 'request_id': 'req-501eda15-7ac3-4326-aeef-475d04aaf2bf'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:53.955156 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-501eda15-7ac3-4326-aeef-475d04aaf2bf admin admin] The token cache is being invalidate because project 53f3a1dc6bbf4cd382c7f507fed85049 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:40:53.955450 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-501eda15-7ac3-4326-aeef-475d04aaf2bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:53.956149 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-501eda15-7ac3-4326-aeef-475d04aaf2bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:53.957826 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1160/2347] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:40:53 2022] DELETE /identity/v3/projects/53f3a1dc6bbf4cd382c7f507fed85049 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:40:53.963586 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e53458-bb12-4c32-a7d7-0f107cec674d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:53.964048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e53458-bb12-4c32-a7d7-0f107cec674d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:53.964222 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e53458-bb12-4c32-a7d7-0f107cec674d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:53.987360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-93e53458-bb12-4c32-a7d7-0f107cec674d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:53.988822 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-93e53458-bb12-4c32-a7d7-0f107cec674d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:54.040804 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:54.065156 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1188/2348] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:54.068770 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265756e-f40f-4357-9669-027635233182 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:54.069416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265756e-f40f-4357-9669-027635233182 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:54.069416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265756e-f40f-4357-9669-027635233182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:54.091649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3265756e-f40f-4357-9669-027635233182 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:54.093082 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3265756e-f40f-4357-9669-027635233182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:54.109343 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1161/2349] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:40:54.112265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e53458-bb12-4c32-a7d7-0f107cec674d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:54.123231 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e53458-bb12-4c32-a7d7-0f107cec674d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:54.130022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28607345-3fc1-46dd-9101-92f00de97c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:54.132619 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28607345-3fc1-46dd-9101-92f00de97c4c admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:54.132619 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28607345-3fc1-46dd-9101-92f00de97c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:54.132619 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28607345-3fc1-46dd-9101-92f00de97c4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:54.133872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28607345-3fc1-46dd-9101-92f00de97c4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:40:54.134789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28607345-3fc1-46dd-9101-92f00de97c4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:54.137368 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28607345-3fc1-46dd-9101-92f00de97c4c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:54.150728 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1189/2350] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:40:54.154622 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265756e-f40f-4357-9669-027635233182 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:54.165121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265756e-f40f-4357-9669-027635233182 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:54.172153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d783066-2e52-4b22-8796-dca58d3d2496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:54.179151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d783066-2e52-4b22-8796-dca58d3d2496 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:54.179151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d783066-2e52-4b22-8796-dca58d3d2496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:54.179151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d783066-2e52-4b22-8796-dca58d3d2496 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:54.179151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d783066-2e52-4b22-8796-dca58d3d2496 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:54.179151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d783066-2e52-4b22-8796-dca58d3d2496 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:54.307103 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1162/2351] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:40:54 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:40:54.309240 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28607345-3fc1-46dd-9101-92f00de97c4c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:54.320812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28607345-3fc1-46dd-9101-92f00de97c4c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:54.344005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d783066-2e52-4b22-8796-dca58d3d2496 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:54.344005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1d783066-2e52-4b22-8796-dca58d3d2496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:54.348957 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:54.403200 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e84d68-5f83-41f9-a9e7-5a243165750d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:54.403794 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e84d68-5f83-41f9-a9e7-5a243165750d admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:54.404078 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e84d68-5f83-41f9-a9e7-5a243165750d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:54.404506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e84d68-5f83-41f9-a9e7-5a243165750d admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:54.407293 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e84d68-5f83-41f9-a9e7-5a243165750d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:54.407689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e84d68-5f83-41f9-a9e7-5a243165750d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:54.421164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d783066-2e52-4b22-8796-dca58d3d2496 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:54.427548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7e603-8488-4be2-9943-092222551ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:54.427548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7e603-8488-4be2-9943-092222551ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:54.427548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7e603-8488-4be2-9943-092222551ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:54.427548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7e603-8488-4be2-9943-092222551ed9 admin admin] PATH_INFO: `/v3/users/3f9edf9cac334601badb5df2c77cac42` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:54.428396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a7e603-8488-4be2-9943-092222551ed9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f9edf9cac334601badb5df2c77cac42)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:54.430349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a7e603-8488-4be2-9943-092222551ed9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:54.493450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1190/2352] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:40:54 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:40:54.497229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e84d68-5f83-41f9-a9e7-5a243165750d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:54.513997 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e84d68-5f83-41f9-a9e7-5a243165750d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:54.541785 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154ce32-582f-40b8-b29b-e7efe8b22d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:54.542556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154ce32-582f-40b8-b29b-e7efe8b22d8e admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:54.542844 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154ce32-582f-40b8-b29b-e7efe8b22d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:54.543125 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154ce32-582f-40b8-b29b-e7efe8b22d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:54.543868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154ce32-582f-40b8-b29b-e7efe8b22d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:54.545535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154ce32-582f-40b8-b29b-e7efe8b22d8e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:54.558401 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1191/2353] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:40:54.562432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154ce32-582f-40b8-b29b-e7efe8b22d8e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:54.575409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154ce32-582f-40b8-b29b-e7efe8b22d8e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:54.580848 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e388b-bd5f-4749-9b56-3a24d4b8e6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:54.581635 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e388b-bd5f-4749-9b56-3a24d4b8e6ee admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:54.581914 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e388b-bd5f-4749-9b56-3a24d4b8e6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:54.582192 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e388b-bd5f-4749-9b56-3a24d4b8e6ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:54.582932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e388b-bd5f-4749-9b56-3a24d4b8e6ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:54.584722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e388b-bd5f-4749-9b56-3a24d4b8e6ee admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:54.597731 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1192/2354] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:40:54.611818 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e388b-bd5f-4749-9b56-3a24d4b8e6ee admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:54.630751 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e388b-bd5f-4749-9b56-3a24d4b8e6ee admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:54.643893 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898aa2b7-4816-4939-8937-dc1dca75a087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:54.646059 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898aa2b7-4816-4939-8937-dc1dca75a087 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:54.646568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898aa2b7-4816-4939-8937-dc1dca75a087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:54.647729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898aa2b7-4816-4939-8937-dc1dca75a087 admin admin] PATH_INFO: `/v3/projects/1e49af480c6c4449b329b5e71904b17c/users/3bbc17dea01a41ad9ca61bc65ed59cf6/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:54.689025 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36a7e603-8488-4be2-9943-092222551ed9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f9edf9cac334601badb5df2c77cac42'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:54.689932 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36a7e603-8488-4be2-9943-092222551ed9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f9edf9cac334601badb5df2c77cac42'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:54.690521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898aa2b7-4816-4939-8937-dc1dca75a087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e49af480c6c4449b329b5e71904b17c, user_id=3bbc17dea01a41ad9ca61bc65ed59cf6, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:54.695850 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898aa2b7-4816-4939-8937-dc1dca75a087 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:54.700414 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36a7e603-8488-4be2-9943-092222551ed9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f9edf9cac334601badb5df2c77cac42'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:54.701455 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36a7e603-8488-4be2-9943-092222551ed9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f9edf9cac334601badb5df2c77cac42'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:54.716259 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36a7e603-8488-4be2-9943-092222551ed9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f9edf9cac334601badb5df2c77cac42'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:54.796912 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1193/2355] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:40:54 2022] PUT /identity/v3/projects/1e49af480c6c4449b329b5e71904b17c/users/3bbc17dea01a41ad9ca61bc65ed59cf6/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:40:54.798497 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdba2d3-7ae7-440a-b033-4f9c5fcfc05d None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:54.798789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdba2d3-7ae7-440a-b033-4f9c5fcfc05d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:54.799070 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdba2d3-7ae7-440a-b033-4f9c5fcfc05d None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:54.800928 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1194/2356] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jun 7 08:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:40:54.807273 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7875cda-1021-43b1-af28-5cfde4c7ccf9 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:54.807577 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7875cda-1021-43b1-af28-5cfde4c7ccf9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:54.807850 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7875cda-1021-43b1-af28-5cfde4c7ccf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:54.821581 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1163/2357] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:40:54 2022] DELETE /identity/v3/users/3f9edf9cac334601badb5df2c77cac42 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:40:54.823369 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203119b-5323-462c-87cd-e8bd388cf67a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:54.823369 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203119b-5323-462c-87cd-e8bd388cf67a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:54.823510 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203119b-5323-462c-87cd-e8bd388cf67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:54.832303 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c7875cda-1021-43b1-af28-5cfde4c7ccf9 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:54.834089 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c7875cda-1021-43b1-af28-5cfde4c7ccf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:54.909285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3203119b-5323-462c-87cd-e8bd388cf67a None None] MFA Rules not processed for user `ee42568b059b49a198ed2c01d5efdafd`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:54.910619 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3203119b-5323-462c-87cd-e8bd388cf67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:54.920623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1195/2358] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:54.921851 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d5089-8832-41ac-a6d7-5c99179ba5e6 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:54.922141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d5089-8832-41ac-a6d7-5c99179ba5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:54.922422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d5089-8832-41ac-a6d7-5c99179ba5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:54.948331 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c33d5089-8832-41ac-a6d7-5c99179ba5e6 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:54.949776 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c33d5089-8832-41ac-a6d7-5c99179ba5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:54.965367 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1196/2359] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:54.966742 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d5089-8832-41ac-a6d7-5c99179ba5e6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:54.968713 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c33d5089-8832-41ac-a6d7-5c99179ba5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:54.981113 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:54.983397 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:55.015726 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1164/2360] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jun 7 08:40:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:55.016968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203119b-5323-462c-87cd-e8bd388cf67a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:55.024094 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3203119b-5323-462c-87cd-e8bd388cf67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:55.067258 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d5089-8832-41ac-a6d7-5c99179ba5e6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:55.070790 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522788ca-da16-4251-8fd5-8aa77df7f1ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:55.071440 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522788ca-da16-4251-8fd5-8aa77df7f1ac service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:55.071720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522788ca-da16-4251-8fd5-8aa77df7f1ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:55.071995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522788ca-da16-4251-8fd5-8aa77df7f1ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:55.082377 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522788ca-da16-4251-8fd5-8aa77df7f1ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:55.083853 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522788ca-da16-4251-8fd5-8aa77df7f1ac service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:55.094719 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203119b-5323-462c-87cd-e8bd388cf67a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:55.095238 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1197/2361] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:40:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:40:55.098567 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0bf4c-384b-40cf-a730-abcf32d396e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:55.099248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0bf4c-384b-40cf-a730-abcf32d396e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:55.099536 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0bf4c-384b-40cf-a730-abcf32d396e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:55.099815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0bf4c-384b-40cf-a730-abcf32d396e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:55.102311 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522788ca-da16-4251-8fd5-8aa77df7f1ac service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:55.103646 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-522788ca-da16-4251-8fd5-8aa77df7f1ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:55.111627 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0bf4c-384b-40cf-a730-abcf32d396e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:55.113848 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0bf4c-384b-40cf-a730-abcf32d396e9 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:55.124963 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1165/2362] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:40:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:40:55.191558 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522788ca-da16-4251-8fd5-8aa77df7f1ac service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:55.195588 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bcbd2-fe89-4607-aa7a-94fa5e367ee1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4d9b5e3826049f2800f1b9424977d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:55.196357 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bcbd2-fe89-4607-aa7a-94fa5e367ee1 service glance] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:55.196634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bcbd2-fe89-4607-aa7a-94fa5e367ee1 service glance] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:55.196921 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bcbd2-fe89-4607-aa7a-94fa5e367ee1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:55.208236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45bcbd2-fe89-4607-aa7a-94fa5e367ee1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:55.210336 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45bcbd2-fe89-4607-aa7a-94fa5e367ee1 service glance] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:55.222462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1198/2363] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:40:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:40:55.462637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc0bc0-f8e1-40e5-a4ab-fec323d02330 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:55.463012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc0bc0-f8e1-40e5-a4ab-fec323d02330 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:55.463354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc0bc0-f8e1-40e5-a4ab-fec323d02330 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:55.466209 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1166/2364] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jun 7 08:40:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:40:55.474466 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3bb00-0021-4321-a3db-8fd0779a6b12 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:55.474879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3bb00-0021-4321-a3db-8fd0779a6b12 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:55.475726 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3bb00-0021-4321-a3db-8fd0779a6b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:55.500299 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a4d3bb00-0021-4321-a3db-8fd0779a6b12 None None] MFA Rules not processed for user `4a87f8bb06ee4a3ba7017422f4f1943e`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:55.500517 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a4d3bb00-0021-4321-a3db-8fd0779a6b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:55.511836 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1199/2365] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jun 7 08:40:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:55.525761 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc0bc0-f8e1-40e5-a4ab-fec323d02330 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:55.527804 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1ffc0bc0-f8e1-40e5-a4ab-fec323d02330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:55.545184 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc0bc0-f8e1-40e5-a4ab-fec323d02330 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:55.549017 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31023d8f-a5af-481f-bdf2-c2c80d91b925 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:55.554101 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31023d8f-a5af-481f-bdf2-c2c80d91b925 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:55.554101 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31023d8f-a5af-481f-bdf2-c2c80d91b925 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:55.554101 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31023d8f-a5af-481f-bdf2-c2c80d91b925 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:55.558508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31023d8f-a5af-481f-bdf2-c2c80d91b925 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:55.560010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31023d8f-a5af-481f-bdf2-c2c80d91b925 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:55.569809 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1167/2366] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:40:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:40:55.582692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3bb00-0021-4321-a3db-8fd0779a6b12 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:55.583338 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a4d3bb00-0021-4321-a3db-8fd0779a6b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:55.601977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3bb00-0021-4321-a3db-8fd0779a6b12 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:55.621984 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7d426-b497-4206-9b78-9af49d5f60cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:55.624074 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7d426-b497-4206-9b78-9af49d5f60cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:55.624720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7d426-b497-4206-9b78-9af49d5f60cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:55.625354 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7d426-b497-4206-9b78-9af49d5f60cd admin admin] PATH_INFO: `/v3/projects/083e79f9fe2a412c95b456e5bc6ff93e` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:55.635414 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7d426-b497-4206-9b78-9af49d5f60cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=083e79f9fe2a412c95b456e5bc6ff93e)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:55.636642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7d426-b497-4206-9b78-9af49d5f60cd admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:55.686706 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e92eb-1a00-4c3f-9604-b336bc6f7ae9 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:55.687226 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e92eb-1a00-4c3f-9604-b336bc6f7ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:55.687498 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e92eb-1a00-4c3f-9604-b336bc6f7ae9 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:55.689070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1168/2367] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jun 7 08:40:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 08:40:55.695561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d704a1-c523-483b-bc85-607e4dbe0169 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:55.695855 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d704a1-c523-483b-bc85-607e4dbe0169 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:55.696227 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d704a1-c523-483b-bc85-607e4dbe0169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:55.718524 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-56d704a1-c523-483b-bc85-607e4dbe0169 None None] MFA Rules not processed for user `ee42568b059b49a198ed2c01d5efdafd`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:55.719768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-56d704a1-c523-483b-bc85-607e4dbe0169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:55.729625 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1169/2368] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jun 7 08:40:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:55.743402 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-83c7d426-b497-4206-9b78-9af49d5f60cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '083e79f9fe2a412c95b456e5bc6ff93e', 'request_id': 'req-83c7d426-b497-4206-9b78-9af49d5f60cd'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:55.758704 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-83c7d426-b497-4206-9b78-9af49d5f60cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '083e79f9fe2a412c95b456e5bc6ff93e', 'request_id': 'req-83c7d426-b497-4206-9b78-9af49d5f60cd'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:55.760174 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d704a1-c523-483b-bc85-607e4dbe0169 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:55.774419 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d704a1-c523-483b-bc85-607e4dbe0169 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:55.843573 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28735eaa-60e4-470e-89cd-04bbd4a086c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:55.844341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28735eaa-60e4-470e-89cd-04bbd4a086c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:55.844686 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28735eaa-60e4-470e-89cd-04bbd4a086c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:55.845203 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28735eaa-60e4-470e-89cd-04bbd4a086c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:55.857735 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-83c7d426-b497-4206-9b78-9af49d5f60cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '083e79f9fe2a412c95b456e5bc6ff93e', 'request_id': 'req-83c7d426-b497-4206-9b78-9af49d5f60cd'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:55.872983 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-83c7d426-b497-4206-9b78-9af49d5f60cd admin admin] The token cache is being invalidate because project 083e79f9fe2a412c95b456e5bc6ff93e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:40:55.873378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-83c7d426-b497-4206-9b78-9af49d5f60cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:55.874517 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-83c7d426-b497-4206-9b78-9af49d5f60cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:55.877858 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1200/2369] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:40:55 2022] DELETE /identity/v3/projects/083e79f9fe2a412c95b456e5bc6ff93e => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:40:55.880767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7d426-b497-4206-9b78-9af49d5f60cd admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:55.882820 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-83c7d426-b497-4206-9b78-9af49d5f60cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:55.898681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:55.930956 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28735eaa-60e4-470e-89cd-04bbd4a086c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:55.933091 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28735eaa-60e4-470e-89cd-04bbd4a086c4 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:55.945772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1170/2370] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:40:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:40:55.957824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7d426-b497-4206-9b78-9af49d5f60cd admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:55.965930 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20751d69-fc52-46e0-b8be-f09f174f7a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:55.966844 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20751d69-fc52-46e0-b8be-f09f174f7a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:55.967125 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20751d69-fc52-46e0-b8be-f09f174f7a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:55.967457 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20751d69-fc52-46e0-b8be-f09f174f7a03 admin admin] PATH_INFO: `/v3/users/1b199b01e3554bb38f747125a7250593` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:55.968783 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20751d69-fc52-46e0-b8be-f09f174f7a03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b199b01e3554bb38f747125a7250593)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:55.970819 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20751d69-fc52-46e0-b8be-f09f174f7a03 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:56.204300 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-20751d69-fc52-46e0-b8be-f09f174f7a03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b199b01e3554bb38f747125a7250593'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:56.205041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-20751d69-fc52-46e0-b8be-f09f174f7a03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b199b01e3554bb38f747125a7250593'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:56.223413 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-20751d69-fc52-46e0-b8be-f09f174f7a03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b199b01e3554bb38f747125a7250593'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:56.223413 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-20751d69-fc52-46e0-b8be-f09f174f7a03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b199b01e3554bb38f747125a7250593'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:56.322462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-20751d69-fc52-46e0-b8be-f09f174f7a03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b199b01e3554bb38f747125a7250593'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:56.342395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1201/2371] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:40:55 2022] DELETE /identity/v3/users/1b199b01e3554bb38f747125a7250593 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:40:56.844296 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28735eaa-60e4-470e-89cd-04bbd4a086c4 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:56.855922 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:56.863538 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28735eaa-60e4-470e-89cd-04bbd4a086c4 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:56.942788 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646b84c-c5f9-47c5-a16d-e2a19b5e3eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:56.943665 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646b84c-c5f9-47c5-a16d-e2a19b5e3eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:56.944614 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646b84c-c5f9-47c5-a16d-e2a19b5e3eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:56.944899 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646b84c-c5f9-47c5-a16d-e2a19b5e3eae admin admin] PATH_INFO: `/v3/projects/543d059d266a45e189115b68bb4effcb` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:56.946345 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a646b84c-c5f9-47c5-a16d-e2a19b5e3eae admin admin] RBAC: Authorizing `identity:delete_project(project_id=543d059d266a45e189115b68bb4effcb)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:56.948649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a646b84c-c5f9-47c5-a16d-e2a19b5e3eae admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:57.106028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a646b84c-c5f9-47c5-a16d-e2a19b5e3eae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '543d059d266a45e189115b68bb4effcb', 'request_id': 'req-a646b84c-c5f9-47c5-a16d-e2a19b5e3eae'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:57.133960 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a646b84c-c5f9-47c5-a16d-e2a19b5e3eae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '543d059d266a45e189115b68bb4effcb', 'request_id': 'req-a646b84c-c5f9-47c5-a16d-e2a19b5e3eae'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:57.141340 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a646b84c-c5f9-47c5-a16d-e2a19b5e3eae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '543d059d266a45e189115b68bb4effcb', 'request_id': 'req-a646b84c-c5f9-47c5-a16d-e2a19b5e3eae'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:57.216731 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a646b84c-c5f9-47c5-a16d-e2a19b5e3eae admin admin] The token cache is being invalidate because project 543d059d266a45e189115b68bb4effcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:40:57.216731 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a646b84c-c5f9-47c5-a16d-e2a19b5e3eae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:57.217578 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a646b84c-c5f9-47c5-a16d-e2a19b5e3eae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:57.220083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1171/2372] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:40:56 2022] DELETE /identity/v3/projects/543d059d266a45e189115b68bb4effcb => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:40:57.224258 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20751d69-fc52-46e0-b8be-f09f174f7a03 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:57.227550 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-20751d69-fc52-46e0-b8be-f09f174f7a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:57.248386 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20751d69-fc52-46e0-b8be-f09f174f7a03 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:57.323405 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999be726-0a89-4157-be66-ed76f74e5dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:57.324699 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999be726-0a89-4157-be66-ed76f74e5dfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:57.325305 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999be726-0a89-4157-be66-ed76f74e5dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:57.325650 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999be726-0a89-4157-be66-ed76f74e5dfa admin admin] PATH_INFO: `/v3/users/e60ab3bc7f0943a4a654268cdde6f0a6` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:57.326964 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999be726-0a89-4157-be66-ed76f74e5dfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e60ab3bc7f0943a4a654268cdde6f0a6)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:57.328973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999be726-0a89-4157-be66-ed76f74e5dfa admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:57.523196 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:57.542297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-999be726-0a89-4157-be66-ed76f74e5dfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e60ab3bc7f0943a4a654268cdde6f0a6'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:57.542951 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-999be726-0a89-4157-be66-ed76f74e5dfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e60ab3bc7f0943a4a654268cdde6f0a6'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:57.555022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-999be726-0a89-4157-be66-ed76f74e5dfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e60ab3bc7f0943a4a654268cdde6f0a6'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:57.555868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-999be726-0a89-4157-be66-ed76f74e5dfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e60ab3bc7f0943a4a654268cdde6f0a6'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:57.595807 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-999be726-0a89-4157-be66-ed76f74e5dfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e60ab3bc7f0943a4a654268cdde6f0a6'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:57.627640 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1202/2373] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:40:57 2022] DELETE /identity/v3/users/e60ab3bc7f0943a4a654268cdde6f0a6 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:40:58.219876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646b84c-c5f9-47c5-a16d-e2a19b5e3eae admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:58.221630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a646b84c-c5f9-47c5-a16d-e2a19b5e3eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:58.252881 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646b84c-c5f9-47c5-a16d-e2a19b5e3eae admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:58.396333 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10718f5-d645-4d3f-9399-2675548a73d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:58.399991 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10718f5-d645-4d3f-9399-2675548a73d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:58.400542 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10718f5-d645-4d3f-9399-2675548a73d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:58.400967 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10718f5-d645-4d3f-9399-2675548a73d6 admin admin] PATH_INFO: `/v3/projects/f6a021fad0b2429f9a98885fa262ad10` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:58.402628 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10718f5-d645-4d3f-9399-2675548a73d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6a021fad0b2429f9a98885fa262ad10)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:58.404784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10718f5-d645-4d3f-9399-2675548a73d6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:58.548312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:58.560224 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f10718f5-d645-4d3f-9399-2675548a73d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6a021fad0b2429f9a98885fa262ad10', 'request_id': 'req-f10718f5-d645-4d3f-9399-2675548a73d6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:58.567597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f10718f5-d645-4d3f-9399-2675548a73d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6a021fad0b2429f9a98885fa262ad10', 'request_id': 'req-f10718f5-d645-4d3f-9399-2675548a73d6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:58.576400 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f10718f5-d645-4d3f-9399-2675548a73d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6a021fad0b2429f9a98885fa262ad10', 'request_id': 'req-f10718f5-d645-4d3f-9399-2675548a73d6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:58.653760 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f10718f5-d645-4d3f-9399-2675548a73d6 admin admin] The token cache is being invalidate because project f6a021fad0b2429f9a98885fa262ad10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:40:58.654047 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f10718f5-d645-4d3f-9399-2675548a73d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:58.654725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f10718f5-d645-4d3f-9399-2675548a73d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:40:58.657647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1172/2374] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:40:58 2022] DELETE /identity/v3/projects/f6a021fad0b2429f9a98885fa262ad10 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:40:58.662306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd0b90-127c-48a4-b17d-ca16f4cd893d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:58.663366 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd0b90-127c-48a4-b17d-ca16f4cd893d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:58.663610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd0b90-127c-48a4-b17d-ca16f4cd893d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:58.684010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7efd0b90-127c-48a4-b17d-ca16f4cd893d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:58.686833 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7efd0b90-127c-48a4-b17d-ca16f4cd893d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:58.777954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1203/2375] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:58.786647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8bb8a-f29d-45bb-a492-f5656838cabe None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:58.786936 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8bb8a-f29d-45bb-a492-f5656838cabe None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:58.787216 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8bb8a-f29d-45bb-a492-f5656838cabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:58.816877 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7be8bb8a-f29d-45bb-a492-f5656838cabe None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:58.816877 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7be8bb8a-f29d-45bb-a492-f5656838cabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:58.826407 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1173/2376] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:58.828601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd0b90-127c-48a4-b17d-ca16f4cd893d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:58.842898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd0b90-127c-48a4-b17d-ca16f4cd893d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:58.855446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7d669-29dc-42c4-a671-a0ee789e5dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:58.856404 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7d669-29dc-42c4-a671-a0ee789e5dea admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:58.856742 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7d669-29dc-42c4-a671-a0ee789e5dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:58.856974 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7d669-29dc-42c4-a671-a0ee789e5dea admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:58.857763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e7d669-29dc-42c4-a671-a0ee789e5dea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:40:58.858079 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e7d669-29dc-42c4-a671-a0ee789e5dea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:58.860030 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e7d669-29dc-42c4-a671-a0ee789e5dea admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:58.877121 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1204/2377] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:40:58.882636 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8bb8a-f29d-45bb-a492-f5656838cabe None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:58.902979 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8bb8a-f29d-45bb-a492-f5656838cabe None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:58.918479 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa011bf-0a15-4885-b56a-7e13581048aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:58.919178 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa011bf-0a15-4885-b56a-7e13581048aa admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:58.919600 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa011bf-0a15-4885-b56a-7e13581048aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:58.919884 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa011bf-0a15-4885-b56a-7e13581048aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:58.920934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa011bf-0a15-4885-b56a-7e13581048aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:58.923424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa011bf-0a15-4885-b56a-7e13581048aa admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:59.026930 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8491f-8a8c-4a4b-a6fc-f9c909b0d6c8 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:59.027224 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8491f-8a8c-4a4b-a6fc-f9c909b0d6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:59.027487 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8491f-8a8c-4a4b-a6fc-f9c909b0d6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:59.031311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1174/2378] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:40:58 2022] POST /identity/v3/projects => generated 389 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:40:59.040290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa011bf-0a15-4885-b56a-7e13581048aa admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:59.103441 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-abd8491f-8a8c-4a4b-a6fc-f9c909b0d6c8 None None] MFA Rules not processed for user `bbf0ce396885472bb5d225aa009043f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:59.105348 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-abd8491f-8a8c-4a4b-a6fc-f9c909b0d6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:59.129894 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa011bf-0a15-4885-b56a-7e13581048aa admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:59.143746 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72095d1b-53e7-4790-8640-d920c1f453b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:59.144448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72095d1b-53e7-4790-8640-d920c1f453b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:59.144733 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72095d1b-53e7-4790-8640-d920c1f453b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:59.145048 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72095d1b-53e7-4790-8640-d920c1f453b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:59.145729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72095d1b-53e7-4790-8640-d920c1f453b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:59.148429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72095d1b-53e7-4790-8640-d920c1f453b7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:59.220165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1205/2379] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jun 7 08:40:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:40:59.233359 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8491f-8a8c-4a4b-a6fc-f9c909b0d6c8 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:59.239556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-abd8491f-8a8c-4a4b-a6fc-f9c909b0d6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:59.249348 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:59.265795 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1175/2380] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:40:59 2022] POST /identity/v3/users => generated 295 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:40:59.269810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72095d1b-53e7-4790-8640-d920c1f453b7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:59.282246 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72095d1b-53e7-4790-8640-d920c1f453b7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:59.288462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cdd54b-0f07-403a-9e5a-ad2d5574f127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:59.289477 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdd54b-0f07-403a-9e5a-ad2d5574f127 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:59.289814 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdd54b-0f07-403a-9e5a-ad2d5574f127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:59.290264 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdd54b-0f07-403a-9e5a-ad2d5574f127 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:59.291012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cdd54b-0f07-403a-9e5a-ad2d5574f127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:59.292752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cdd54b-0f07-403a-9e5a-ad2d5574f127 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:59.305102 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1176/2381] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:40:59.311143 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cdd54b-0f07-403a-9e5a-ad2d5574f127 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:59.325375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cdd54b-0f07-403a-9e5a-ad2d5574f127 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:59.330807 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884d821-6f19-4049-b070-5faa7e2189ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:59.331741 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884d821-6f19-4049-b070-5faa7e2189ac admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:59.332215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884d821-6f19-4049-b070-5faa7e2189ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:59.332668 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884d821-6f19-4049-b070-5faa7e2189ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:59.333480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884d821-6f19-4049-b070-5faa7e2189ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:59.336427 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884d821-6f19-4049-b070-5faa7e2189ac admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:59.352372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1177/2382] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:40:59.353249 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884d821-6f19-4049-b070-5faa7e2189ac admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:59.364346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884d821-6f19-4049-b070-5faa7e2189ac admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:59.369042 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8491f-8a8c-4a4b-a6fc-f9c909b0d6c8 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:59.370423 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265217b-fe9c-4bac-b258-f690d888b1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:59.371375 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265217b-fe9c-4bac-b258-f690d888b1d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:59.371700 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265217b-fe9c-4bac-b258-f690d888b1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:59.371981 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265217b-fe9c-4bac-b258-f690d888b1d6 admin admin] PATH_INFO: `/v3/projects/e656b72d62de481f95fcd7c732445d32/users/ef9f9c6d85424c5493e6a1c77075e9f9/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:59.375370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d0f9a-c82b-4219-aeea-1ee59c9453d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334dc47ae2e643ccb44d25c6b5906f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:59.376163 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d0f9a-c82b-4219-aeea-1ee59c9453d2 service swift] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:59.379460 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d0f9a-c82b-4219-aeea-1ee59c9453d2 service swift] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:59.379460 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d0f9a-c82b-4219-aeea-1ee59c9453d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:59.395752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265217b-fe9c-4bac-b258-f690d888b1d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e656b72d62de481f95fcd7c732445d32, user_id=ef9f9c6d85424c5493e6a1c77075e9f9, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:59.396478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d0f9a-c82b-4219-aeea-1ee59c9453d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:59.398904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d0f9a-c82b-4219-aeea-1ee59c9453d2 service swift] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:59.400190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265217b-fe9c-4bac-b258-f690d888b1d6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:59.421116 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1206/2383] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jun 7 08:40:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 08:40:59.543521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1178/2384] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:40:59 2022] PUT /identity/v3/projects/e656b72d62de481f95fcd7c732445d32/users/ef9f9c6d85424c5493e6a1c77075e9f9/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:40:59.553940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b99ab-7542-418b-b55a-4355df1b85de None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:59.554228 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b99ab-7542-418b-b55a-4355df1b85de None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:59.554501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b99ab-7542-418b-b55a-4355df1b85de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:59.656940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-066b99ab-7542-418b-b55a-4355df1b85de None None] MFA Rules not processed for user `ef9f9c6d85424c5493e6a1c77075e9f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:40:59.659488 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-066b99ab-7542-418b-b55a-4355df1b85de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:59.741085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1207/2385] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:40:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:40:59.754064 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265217b-fe9c-4bac-b258-f690d888b1d6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:40:59.758231 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d265217b-fe9c-4bac-b258-f690d888b1d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:40:59.862055 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:40:59.898435 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265217b-fe9c-4bac-b258-f690d888b1d6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:40:59.904051 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd3922-b60a-4b8d-872b-bdb9a419503c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:40:59.905637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd3922-b60a-4b8d-872b-bdb9a419503c service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:40:59.906937 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd3922-b60a-4b8d-872b-bdb9a419503c service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:40:59.907992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd3922-b60a-4b8d-872b-bdb9a419503c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:40:59.929363 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd3922-b60a-4b8d-872b-bdb9a419503c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:40:59.931156 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd3922-b60a-4b8d-872b-bdb9a419503c service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:40:59.965203 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1179/2386] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:40:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:41:02.459525 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f5435a-a395-44d5-8ba2-f5db5a2b943e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:02.459946 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f5435a-a395-44d5-8ba2-f5db5a2b943e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:02.460227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f5435a-a395-44d5-8ba2-f5db5a2b943e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:02.521426 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-f8f5435a-a395-44d5-8ba2-f5db5a2b943e None None] MFA Rules not processed for user `3bbc17dea01a41ad9ca61bc65ed59cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:41:02.522981 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f8f5435a-a395-44d5-8ba2-f5db5a2b943e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:02.695658 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:02.704261 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1208/2387] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:41:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:41:02.709193 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd3922-b60a-4b8d-872b-bdb9a419503c service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:02.711966 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7fdd3922-b60a-4b8d-872b-bdb9a419503c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:02.876162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd3922-b60a-4b8d-872b-bdb9a419503c service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:02.884657 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32715fa6-72a5-47ca-9544-513d617ee6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:02.885888 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32715fa6-72a5-47ca-9544-513d617ee6be admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:02.891126 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32715fa6-72a5-47ca-9544-513d617ee6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:02.891126 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32715fa6-72a5-47ca-9544-513d617ee6be admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:02.891126 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32715fa6-72a5-47ca-9544-513d617ee6be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:02.891126 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32715fa6-72a5-47ca-9544-513d617ee6be admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:03.215180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1180/2388] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:41:02 2022] POST /identity/v3/projects => generated 385 bytes in 506 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:41:03.216212 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f5435a-a395-44d5-8ba2-f5db5a2b943e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:03.236294 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f5435a-a395-44d5-8ba2-f5db5a2b943e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:03.327138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f9823-35bf-42c2-bb0b-27795d9d6849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:03.327710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f9823-35bf-42c2-bb0b-27795d9d6849 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:03.328094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f9823-35bf-42c2-bb0b-27795d9d6849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:03.328406 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f9823-35bf-42c2-bb0b-27795d9d6849 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:03.329163 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f9823-35bf-42c2-bb0b-27795d9d6849 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:03.331536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f9823-35bf-42c2-bb0b-27795d9d6849 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:03.473351 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1209/2389] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:41:03 2022] POST /identity/v3/users => generated 299 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:41:03.476085 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32715fa6-72a5-47ca-9544-513d617ee6be admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:03.491706 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32715fa6-72a5-47ca-9544-513d617ee6be admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:03.496617 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e24ee6-f47a-48b6-a41a-03a7779f55e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:03.497668 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e24ee6-f47a-48b6-a41a-03a7779f55e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:03.499326 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e24ee6-f47a-48b6-a41a-03a7779f55e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:03.499795 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e24ee6-f47a-48b6-a41a-03a7779f55e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:03.502255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e24ee6-f47a-48b6-a41a-03a7779f55e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:03.504193 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e24ee6-f47a-48b6-a41a-03a7779f55e7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:03.529888 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1181/2390] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:41:03.536132 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f9823-35bf-42c2-bb0b-27795d9d6849 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:03.550712 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f9823-35bf-42c2-bb0b-27795d9d6849 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:03.555971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52cb0e-20af-40c2-8ae2-8a15ca1e1ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:03.557291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52cb0e-20af-40c2-8ae2-8a15ca1e1ae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:03.557894 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52cb0e-20af-40c2-8ae2-8a15ca1e1ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:03.558390 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52cb0e-20af-40c2-8ae2-8a15ca1e1ae4 admin admin] PATH_INFO: `/v3/projects/74fcd48e72144c8280157e95247c5f47/users/11f8e58d1a234cbfa71d799541c70a4c/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:03.579952 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52cb0e-20af-40c2-8ae2-8a15ca1e1ae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74fcd48e72144c8280157e95247c5f47, user_id=11f8e58d1a234cbfa71d799541c70a4c, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:03.582997 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52cb0e-20af-40c2-8ae2-8a15ca1e1ae4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:03.657582 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1210/2391] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:41:03 2022] PUT /identity/v3/projects/74fcd48e72144c8280157e95247c5f47/users/11f8e58d1a234cbfa71d799541c70a4c/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:41:03.661385 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e24ee6-f47a-48b6-a41a-03a7779f55e7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:03.677754 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e24ee6-f47a-48b6-a41a-03a7779f55e7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:03.794299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447ef4b-e6f8-4732-9ed1-16dc1da1d229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:03.794995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447ef4b-e6f8-4732-9ed1-16dc1da1d229 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:03.795257 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447ef4b-e6f8-4732-9ed1-16dc1da1d229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:03.795532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447ef4b-e6f8-4732-9ed1-16dc1da1d229 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:03.796559 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447ef4b-e6f8-4732-9ed1-16dc1da1d229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:03.798272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447ef4b-e6f8-4732-9ed1-16dc1da1d229 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:03.818000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1182/2392] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:41:03.818728 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52cb0e-20af-40c2-8ae2-8a15ca1e1ae4 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:03.840328 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52cb0e-20af-40c2-8ae2-8a15ca1e1ae4 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:03.850848 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ee2d5-4634-4fd1-930f-cb2518b67650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:03.852102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ee2d5-4634-4fd1-930f-cb2518b67650 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:03.852378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ee2d5-4634-4fd1-930f-cb2518b67650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:03.852662 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ee2d5-4634-4fd1-930f-cb2518b67650 admin admin] PATH_INFO: `/v3/projects/74fcd48e72144c8280157e95247c5f47/users/11f8e58d1a234cbfa71d799541c70a4c/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:03.859405 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ee2d5-4634-4fd1-930f-cb2518b67650 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74fcd48e72144c8280157e95247c5f47, user_id=11f8e58d1a234cbfa71d799541c70a4c, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:03.864197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ee2d5-4634-4fd1-930f-cb2518b67650 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:03.886593 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1211/2393] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:41:03 2022] PUT /identity/v3/projects/74fcd48e72144c8280157e95247c5f47/users/11f8e58d1a234cbfa71d799541c70a4c/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:41:11.047705 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447ef4b-e6f8-4732-9ed1-16dc1da1d229 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:11.050253 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9447ef4b-e6f8-4732-9ed1-16dc1da1d229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:11.148100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:11.230053 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447ef4b-e6f8-4732-9ed1-16dc1da1d229 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:11.423215 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7556c3b-def2-4e81-be63-2a4c7e37ce72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc0d7bd7e4547f4a8d90ffc2c2e1d03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:11.423924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7556c3b-def2-4e81-be63-2a4c7e37ce72 service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:11.424203 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7556c3b-def2-4e81-be63-2a4c7e37ce72 service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:11.425274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7556c3b-def2-4e81-be63-2a4c7e37ce72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:11.431268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f7556c3b-def2-4e81-be63-2a4c7e37ce72 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:11.468795 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7556c3b-def2-4e81-be63-2a4c7e37ce72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:11.470030 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7556c3b-def2-4e81-be63-2a4c7e37ce72 service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:11.486479 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1183/2394] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:41:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:41:13.194515 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a4534-7810-4a3c-b70b-00f6f2422705 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:13.195416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a4534-7810-4a3c-b70b-00f6f2422705 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:13.195862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a4534-7810-4a3c-b70b-00f6f2422705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:13.231791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-0f9a4534-7810-4a3c-b70b-00f6f2422705 None None] MFA Rules not processed for user `11f8e58d1a234cbfa71d799541c70a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:41:13.233100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0f9a4534-7810-4a3c-b70b-00f6f2422705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:13.335582 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1212/2395] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:41:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:41:13.349358 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7556c3b-def2-4e81-be63-2a4c7e37ce72 service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:13.351513 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f7556c3b-def2-4e81-be63-2a4c7e37ce72 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:13.466899 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7556c3b-def2-4e81-be63-2a4c7e37ce72 service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:13.476404 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974871a-5559-47fd-9586-6f63d4b36a84 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4d9b5e3826049f2800f1b9424977d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:13.477199 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974871a-5559-47fd-9586-6f63d4b36a84 service glance] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:13.477535 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974871a-5559-47fd-9586-6f63d4b36a84 service glance] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:13.477872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974871a-5559-47fd-9586-6f63d4b36a84 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:13.559504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974871a-5559-47fd-9586-6f63d4b36a84 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:13.561135 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974871a-5559-47fd-9586-6f63d4b36a84 service glance] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:13.597078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1184/2396] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:41:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:41:13.917165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8923ee-0dc9-4af9-a692-74813cd69327 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:13.917165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8923ee-0dc9-4af9-a692-74813cd69327 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:13.917165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8923ee-0dc9-4af9-a692-74813cd69327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:13.959272 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-6a8923ee-0dc9-4af9-a692-74813cd69327 None None] MFA Rules not processed for user `bbf0ce396885472bb5d225aa009043f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:41:13.961158 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6a8923ee-0dc9-4af9-a692-74813cd69327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:14.061940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:14.139537 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1213/2397] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jun 7 08:41:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:41:14.165687 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974871a-5559-47fd-9586-6f63d4b36a84 service glance] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:14.209953 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:14.315806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974871a-5559-47fd-9586-6f63d4b36a84 service glance] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:14.323989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedd4b9-9c59-4fcc-8165-6ecc8ec6b0d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334dc47ae2e643ccb44d25c6b5906f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:14.324794 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedd4b9-9c59-4fcc-8165-6ecc8ec6b0d4 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:14.325173 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedd4b9-9c59-4fcc-8165-6ecc8ec6b0d4 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:14.327357 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedd4b9-9c59-4fcc-8165-6ecc8ec6b0d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:14.351195 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedd4b9-9c59-4fcc-8165-6ecc8ec6b0d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:14.353842 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedd4b9-9c59-4fcc-8165-6ecc8ec6b0d4 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:14.370234 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1185/2398] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jun 7 08:41:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 08:41:23.668035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8923ee-0dc9-4af9-a692-74813cd69327 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:23.735813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8923ee-0dc9-4af9-a692-74813cd69327 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:23.741784 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33f74a-4ac6-4e74-82b0-0e296d9260f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:23.742703 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33f74a-4ac6-4e74-82b0-0e296d9260f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:23.742980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33f74a-4ac6-4e74-82b0-0e296d9260f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:23.743247 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33f74a-4ac6-4e74-82b0-0e296d9260f4 admin admin] PATH_INFO: `/v3/users/ef9f9c6d85424c5493e6a1c77075e9f9` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:23.744441 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33f74a-4ac6-4e74-82b0-0e296d9260f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef9f9c6d85424c5493e6a1c77075e9f9)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:23.746416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33f74a-4ac6-4e74-82b0-0e296d9260f4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:24.097446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ba33f74a-4ac6-4e74-82b0-0e296d9260f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef9f9c6d85424c5493e6a1c77075e9f9'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:24.098107 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ba33f74a-4ac6-4e74-82b0-0e296d9260f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef9f9c6d85424c5493e6a1c77075e9f9'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:24.114849 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ba33f74a-4ac6-4e74-82b0-0e296d9260f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef9f9c6d85424c5493e6a1c77075e9f9'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:24.114849 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ba33f74a-4ac6-4e74-82b0-0e296d9260f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef9f9c6d85424c5493e6a1c77075e9f9'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:24.226828 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ba33f74a-4ac6-4e74-82b0-0e296d9260f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef9f9c6d85424c5493e6a1c77075e9f9'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:24.256552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1214/2399] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:41:23 2022] DELETE /identity/v3/users/ef9f9c6d85424c5493e6a1c77075e9f9 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:41:24.261187 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144172c-90bd-4ed0-af3c-fca99eb2fb2a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:24.261918 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144172c-90bd-4ed0-af3c-fca99eb2fb2a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:24.261918 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144172c-90bd-4ed0-af3c-fca99eb2fb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:24.293960 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0144172c-90bd-4ed0-af3c-fca99eb2fb2a None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:41:24.297115 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0144172c-90bd-4ed0-af3c-fca99eb2fb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:24.498842 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1186/2400] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:41:24.508678 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33f74a-4ac6-4e74-82b0-0e296d9260f4 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:24.510736 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ba33f74a-4ac6-4e74-82b0-0e296d9260f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:24.630457 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33f74a-4ac6-4e74-82b0-0e296d9260f4 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:24.639148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce40ab-fd77-46ce-9736-191fca990ce5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:24.641425 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce40ab-fd77-46ce-9736-191fca990ce5 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:24.641904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce40ab-fd77-46ce-9736-191fca990ce5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:24.642352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce40ab-fd77-46ce-9736-191fca990ce5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:24.667646 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce40ab-fd77-46ce-9736-191fca990ce5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:24.669484 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce40ab-fd77-46ce-9736-191fca990ce5 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:24.681302 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1215/2401] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:41:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:41:25.357858 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144172c-90bd-4ed0-af3c-fca99eb2fb2a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:25.362817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0144172c-90bd-4ed0-af3c-fca99eb2fb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:25.415725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144172c-90bd-4ed0-af3c-fca99eb2fb2a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:25.452119 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933941d-fa45-4134-9a53-e5c77e5e668b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:25.453087 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933941d-fa45-4134-9a53-e5c77e5e668b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:25.453371 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933941d-fa45-4134-9a53-e5c77e5e668b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:25.453693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933941d-fa45-4134-9a53-e5c77e5e668b admin admin] PATH_INFO: `/v3/projects/e656b72d62de481f95fcd7c732445d32` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:25.455147 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933941d-fa45-4134-9a53-e5c77e5e668b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e656b72d62de481f95fcd7c732445d32)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:25.457052 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933941d-fa45-4134-9a53-e5c77e5e668b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:25.579515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0933941d-fa45-4134-9a53-e5c77e5e668b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e656b72d62de481f95fcd7c732445d32', 'request_id': 'req-0933941d-fa45-4134-9a53-e5c77e5e668b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:25.587207 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0933941d-fa45-4134-9a53-e5c77e5e668b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e656b72d62de481f95fcd7c732445d32', 'request_id': 'req-0933941d-fa45-4134-9a53-e5c77e5e668b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:25.594714 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0933941d-fa45-4134-9a53-e5c77e5e668b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e656b72d62de481f95fcd7c732445d32', 'request_id': 'req-0933941d-fa45-4134-9a53-e5c77e5e668b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:25.624285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a245dc-c96a-43ff-a1b5-ebdbf475a60d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:25.624285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a245dc-c96a-43ff-a1b5-ebdbf475a60d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:25.624610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a245dc-c96a-43ff-a1b5-ebdbf475a60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:25.631945 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:25.648621 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a6a245dc-c96a-43ff-a1b5-ebdbf475a60d None None] MFA Rules not processed for user `bbf0ce396885472bb5d225aa009043f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:41:25.650304 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a6a245dc-c96a-43ff-a1b5-ebdbf475a60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:25.690471 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0933941d-fa45-4134-9a53-e5c77e5e668b admin admin] The token cache is being invalidate because project e656b72d62de481f95fcd7c732445d32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:41:25.691417 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0933941d-fa45-4134-9a53-e5c77e5e668b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:25.692594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0933941d-fa45-4134-9a53-e5c77e5e668b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:25.713774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1187/2402] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:41:25 2022] DELETE /identity/v3/projects/e656b72d62de481f95fcd7c732445d32 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:41:25.757084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1216/2403] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jun 7 08:41:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:41:25.769459 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933941d-fa45-4134-9a53-e5c77e5e668b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:25.770694 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0933941d-fa45-4134-9a53-e5c77e5e668b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:25.844185 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:25.947762 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933941d-fa45-4134-9a53-e5c77e5e668b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:25.955279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83612f5c-737d-4768-a0a6-3f6c8d376ed3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334dc47ae2e643ccb44d25c6b5906f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:25.956407 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83612f5c-737d-4768-a0a6-3f6c8d376ed3 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:25.956892 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83612f5c-737d-4768-a0a6-3f6c8d376ed3 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:25.957360 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83612f5c-737d-4768-a0a6-3f6c8d376ed3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:25.970591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83612f5c-737d-4768-a0a6-3f6c8d376ed3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:25.980092 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83612f5c-737d-4768-a0a6-3f6c8d376ed3 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:25.992248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1188/2404] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jun 7 08:41:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:41:31.158317 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac7dd9-a479-492c-a4dc-ef8f073e75c9 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:31.159592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac7dd9-a479-492c-a4dc-ef8f073e75c9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:31.159592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac7dd9-a479-492c-a4dc-ef8f073e75c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:31.184500 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a8ac7dd9-a479-492c-a4dc-ef8f073e75c9 None None] MFA Rules not processed for user `bbf0ce396885472bb5d225aa009043f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:41:31.186365 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a8ac7dd9-a479-492c-a4dc-ef8f073e75c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:31.199033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1217/2405] 10.222.0.17 () {58 vars in 967 bytes} [Tue Jun 7 08:41:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:41:31.221855 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83612f5c-737d-4768-a0a6-3f6c8d376ed3 service swift] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:31.239149 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83612f5c-737d-4768-a0a6-3f6c8d376ed3 service swift] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:31.252244 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ff66a-3feb-406b-ad7d-8ff7ad8b0033 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334dc47ae2e643ccb44d25c6b5906f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:31.252244 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ff66a-3feb-406b-ad7d-8ff7ad8b0033 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:31.252244 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ff66a-3feb-406b-ad7d-8ff7ad8b0033 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:31.252244 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ff66a-3feb-406b-ad7d-8ff7ad8b0033 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:31.259101 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ff66a-3feb-406b-ad7d-8ff7ad8b0033 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:31.260704 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ff66a-3feb-406b-ad7d-8ff7ad8b0033 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:31.280562 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afc928-292a-4c81-8f30-95d2ed08d244 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:31.281020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afc928-292a-4c81-8f30-95d2ed08d244 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:31.281405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1189/2406] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jun 7 08:41:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:41:31.281751 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afc928-292a-4c81-8f30-95d2ed08d244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:31.303617 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-f7afc928-292a-4c81-8f30-95d2ed08d244 None None] MFA Rules not processed for user `3bbc17dea01a41ad9ca61bc65ed59cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:41:31.305522 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f7afc928-292a-4c81-8f30-95d2ed08d244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:31.373851 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1218/2407] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:41:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:41:36.888815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ff66a-3feb-406b-ad7d-8ff7ad8b0033 service swift] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:36.888815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6c7ff66a-3feb-406b-ad7d-8ff7ad8b0033 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:36.966835 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ff66a-3feb-406b-ad7d-8ff7ad8b0033 service swift] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:36.971608 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bef964-4291-4e31-98e0-1605ca8665e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:36.975202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bef964-4291-4e31-98e0-1605ca8665e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:36.975202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bef964-4291-4e31-98e0-1605ca8665e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:36.975202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bef964-4291-4e31-98e0-1605ca8665e2 admin admin] PATH_INFO: `/v3/users/489306774e664047a2a5629048455686` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:36.975202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bef964-4291-4e31-98e0-1605ca8665e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=489306774e664047a2a5629048455686)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:36.976204 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bef964-4291-4e31-98e0-1605ca8665e2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:37.226775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a4bef964-4291-4e31-98e0-1605ca8665e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '489306774e664047a2a5629048455686'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:37.227469 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a4bef964-4291-4e31-98e0-1605ca8665e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '489306774e664047a2a5629048455686'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:37.235961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a4bef964-4291-4e31-98e0-1605ca8665e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '489306774e664047a2a5629048455686'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:37.236721 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a4bef964-4291-4e31-98e0-1605ca8665e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '489306774e664047a2a5629048455686'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:37.255271 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a4bef964-4291-4e31-98e0-1605ca8665e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '489306774e664047a2a5629048455686'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:37.321882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1190/2408] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:41:36 2022] DELETE /identity/v3/users/489306774e664047a2a5629048455686 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:41:37.613027 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7afc928-292a-4c81-8f30-95d2ed08d244 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:37.625839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7afc928-292a-4c81-8f30-95d2ed08d244 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:37.701025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910a732-daf2-4c07-ab35-f9fe031ccbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:37.701893 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910a732-daf2-4c07-ab35-f9fe031ccbf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:37.702174 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910a732-daf2-4c07-ab35-f9fe031ccbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:37.702450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910a732-daf2-4c07-ab35-f9fe031ccbf7 admin admin] PATH_INFO: `/v3/projects/be66894579564c8584bb7a4ff489ea22` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:37.703636 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910a732-daf2-4c07-ab35-f9fe031ccbf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be66894579564c8584bb7a4ff489ea22)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:37.705816 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910a732-daf2-4c07-ab35-f9fe031ccbf7 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:37.835029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:37.836859 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5910a732-daf2-4c07-ab35-f9fe031ccbf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be66894579564c8584bb7a4ff489ea22', 'request_id': 'req-5910a732-daf2-4c07-ab35-f9fe031ccbf7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:37.843192 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5910a732-daf2-4c07-ab35-f9fe031ccbf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be66894579564c8584bb7a4ff489ea22', 'request_id': 'req-5910a732-daf2-4c07-ab35-f9fe031ccbf7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:37.931067 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5910a732-daf2-4c07-ab35-f9fe031ccbf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be66894579564c8584bb7a4ff489ea22', 'request_id': 'req-5910a732-daf2-4c07-ab35-f9fe031ccbf7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:37.942147 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5910a732-daf2-4c07-ab35-f9fe031ccbf7 admin admin] The token cache is being invalidate because project be66894579564c8584bb7a4ff489ea22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:41:37.942433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5910a732-daf2-4c07-ab35-f9fe031ccbf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:37.943084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5910a732-daf2-4c07-ab35-f9fe031ccbf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:37.945531 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1219/2409] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:41:37 2022] DELETE /identity/v3/projects/be66894579564c8584bb7a4ff489ea22 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:41:37.951457 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89586b71-d7d8-4717-affb-843f63f0c345 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:37.951906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89586b71-d7d8-4717-affb-843f63f0c345 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:37.952147 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89586b71-d7d8-4717-affb-843f63f0c345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:37.973623 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-89586b71-d7d8-4717-affb-843f63f0c345 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:41:37.975012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-89586b71-d7d8-4717-affb-843f63f0c345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:38.002140 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:38.047057 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1191/2410] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:41:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:41:38.051079 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369c205-6cf9-4892-a8af-91c25204f447 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:38.051360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369c205-6cf9-4892-a8af-91c25204f447 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:38.051637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369c205-6cf9-4892-a8af-91c25204f447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:38.069986 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a369c205-6cf9-4892-a8af-91c25204f447 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:41:38.071206 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a369c205-6cf9-4892-a8af-91c25204f447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:38.084015 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1220/2411] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:41:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:41:38.089027 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89586b71-d7d8-4717-affb-843f63f0c345 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:38.103222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89586b71-d7d8-4717-affb-843f63f0c345 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:38.107919 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f965d-ceed-4d27-8524-3364e72c333b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:38.108587 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f965d-ceed-4d27-8524-3364e72c333b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:38.108869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f965d-ceed-4d27-8524-3364e72c333b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:38.109181 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f965d-ceed-4d27-8524-3364e72c333b admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:38.110005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f965d-ceed-4d27-8524-3364e72c333b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:41:38.110319 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f965d-ceed-4d27-8524-3364e72c333b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:38.111972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f965d-ceed-4d27-8524-3364e72c333b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:38.125328 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1192/2412] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:41:38.130096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369c205-6cf9-4892-a8af-91c25204f447 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:38.140429 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369c205-6cf9-4892-a8af-91c25204f447 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:38.145779 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9049242-a69f-4fd9-9b7e-b68bf3cd1e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:38.146475 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9049242-a69f-4fd9-9b7e-b68bf3cd1e22 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:38.146795 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9049242-a69f-4fd9-9b7e-b68bf3cd1e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:38.148424 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9049242-a69f-4fd9-9b7e-b68bf3cd1e22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:38.148424 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9049242-a69f-4fd9-9b7e-b68bf3cd1e22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:38.150404 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9049242-a69f-4fd9-9b7e-b68bf3cd1e22 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:38.253213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1221/2413] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:41:38 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:41:38.257414 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f965d-ceed-4d27-8524-3364e72c333b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:38.271434 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f965d-ceed-4d27-8524-3364e72c333b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:38.339315 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7486e-cc9c-488d-a89a-2129b295a682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:38.339944 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7486e-cc9c-488d-a89a-2129b295a682 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:38.340232 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7486e-cc9c-488d-a89a-2129b295a682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:38.340512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7486e-cc9c-488d-a89a-2129b295a682 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:38.341249 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7486e-cc9c-488d-a89a-2129b295a682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:38.343283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7486e-cc9c-488d-a89a-2129b295a682 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:38.470322 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1193/2414] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:41:38 2022] POST /identity/v3/users => generated 287 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:41:38.474597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9049242-a69f-4fd9-9b7e-b68bf3cd1e22 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:38.490145 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9049242-a69f-4fd9-9b7e-b68bf3cd1e22 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:38.515032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea4100-9ffd-4fdc-a611-82b820fd76c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:38.515647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea4100-9ffd-4fdc-a611-82b820fd76c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:38.515930 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea4100-9ffd-4fdc-a611-82b820fd76c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:38.516205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea4100-9ffd-4fdc-a611-82b820fd76c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:38.517029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea4100-9ffd-4fdc-a611-82b820fd76c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:38.518807 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea4100-9ffd-4fdc-a611-82b820fd76c4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:38.532165 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1222/2415] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:41:38.536629 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7486e-cc9c-488d-a89a-2129b295a682 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:38.549623 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7486e-cc9c-488d-a89a-2129b295a682 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:38.556509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78693fd4-728d-4b06-899a-1e565c78d5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:38.557213 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78693fd4-728d-4b06-899a-1e565c78d5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:38.557495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78693fd4-728d-4b06-899a-1e565c78d5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:38.557768 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78693fd4-728d-4b06-899a-1e565c78d5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:38.558507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78693fd4-728d-4b06-899a-1e565c78d5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:38.560302 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78693fd4-728d-4b06-899a-1e565c78d5a2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:38.571003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1194/2416] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:41:38.575178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea4100-9ffd-4fdc-a611-82b820fd76c4 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:38.601724 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea4100-9ffd-4fdc-a611-82b820fd76c4 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:38.601724 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf54c58-dfb8-47d7-8627-ef24cc5e277e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:38.601724 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf54c58-dfb8-47d7-8627-ef24cc5e277e admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:38.601724 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf54c58-dfb8-47d7-8627-ef24cc5e277e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:38.602099 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf54c58-dfb8-47d7-8627-ef24cc5e277e admin admin] PATH_INFO: `/v3/projects/5b777747392942a2b7ec1b6aa7d7bafa/users/414928a69dab4a61893cf799bc3cf1a8/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:38.618731 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf54c58-dfb8-47d7-8627-ef24cc5e277e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b777747392942a2b7ec1b6aa7d7bafa, user_id=414928a69dab4a61893cf799bc3cf1a8, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:38.622179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf54c58-dfb8-47d7-8627-ef24cc5e277e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:38.772378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1223/2417] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:41:38 2022] PUT /identity/v3/projects/5b777747392942a2b7ec1b6aa7d7bafa/users/414928a69dab4a61893cf799bc3cf1a8/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:41:38.783385 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07cdf4-2802-4438-b3be-48030e2d7f7a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:38.783519 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07cdf4-2802-4438-b3be-48030e2d7f7a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:38.783844 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07cdf4-2802-4438-b3be-48030e2d7f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:38.813717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9e07cdf4-2802-4438-b3be-48030e2d7f7a None None] MFA Rules not processed for user `414928a69dab4a61893cf799bc3cf1a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:41:38.814454 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9e07cdf4-2802-4438-b3be-48030e2d7f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:38.860193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efa9d9-75a1-4fe8-8c16-89cffcc31480 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:38.860483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efa9d9-75a1-4fe8-8c16-89cffcc31480 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:38.860767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efa9d9-75a1-4fe8-8c16-89cffcc31480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:38.880843 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-62efa9d9-75a1-4fe8-8c16-89cffcc31480 None None] MFA Rules not processed for user `bbf0ce396885472bb5d225aa009043f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:41:38.882726 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-62efa9d9-75a1-4fe8-8c16-89cffcc31480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:38.897370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1195/2418] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:41:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:41:38.908991 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07cdf4-2802-4438-b3be-48030e2d7f7a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:38.911113 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9e07cdf4-2802-4438-b3be-48030e2d7f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:38.949184 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1224/2419] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jun 7 08:41:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:41:38.953934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:38.962797 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efa9d9-75a1-4fe8-8c16-89cffcc31480 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:38.964516 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-62efa9d9-75a1-4fe8-8c16-89cffcc31480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:38.989382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07cdf4-2802-4438-b3be-48030e2d7f7a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:38.993941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf2022-f404-45f2-b166-f87ee365730c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:38.993941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf2022-f404-45f2-b166-f87ee365730c service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:38.994173 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf2022-f404-45f2-b166-f87ee365730c service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:38.994273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf2022-f404-45f2-b166-f87ee365730c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:39.002967 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf2022-f404-45f2-b166-f87ee365730c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:39.004404 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf2022-f404-45f2-b166-f87ee365730c service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:39.014099 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1196/2420] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:41:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:41:39.042408 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf2022-f404-45f2-b166-f87ee365730c service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:39.047037 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efa9d9-75a1-4fe8-8c16-89cffcc31480 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:39.052276 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51330176-54a0-4c20-9e20-acd8cbe53896 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334dc47ae2e643ccb44d25c6b5906f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:39.052951 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51330176-54a0-4c20-9e20-acd8cbe53896 service swift] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:39.053267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51330176-54a0-4c20-9e20-acd8cbe53896 service swift] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:39.053550 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51330176-54a0-4c20-9e20-acd8cbe53896 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:39.065781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51330176-54a0-4c20-9e20-acd8cbe53896 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:39.067715 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51330176-54a0-4c20-9e20-acd8cbe53896 service swift] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:39.079532 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1225/2421] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jun 7 08:41:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 08:41:39.133017 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf2022-f404-45f2-b166-f87ee365730c service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:39.138563 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd3eb2-f81e-4bd0-879a-9dc41e281058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:39.139753 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd3eb2-f81e-4bd0-879a-9dc41e281058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:39.140137 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd3eb2-f81e-4bd0-879a-9dc41e281058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:39.140527 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd3eb2-f81e-4bd0-879a-9dc41e281058 admin admin] PATH_INFO: `/v3/users/414928a69dab4a61893cf799bc3cf1a8` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:39.142626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd3eb2-f81e-4bd0-879a-9dc41e281058 admin admin] RBAC: Authorizing `identity:delete_user(user_id=414928a69dab4a61893cf799bc3cf1a8)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:39.144878 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd3eb2-f81e-4bd0-879a-9dc41e281058 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:39.344692 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ebfd3eb2-f81e-4bd0-879a-9dc41e281058 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '414928a69dab4a61893cf799bc3cf1a8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:39.346288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ebfd3eb2-f81e-4bd0-879a-9dc41e281058 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '414928a69dab4a61893cf799bc3cf1a8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:39.365930 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ebfd3eb2-f81e-4bd0-879a-9dc41e281058 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '414928a69dab4a61893cf799bc3cf1a8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:39.367950 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ebfd3eb2-f81e-4bd0-879a-9dc41e281058 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '414928a69dab4a61893cf799bc3cf1a8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:39.399993 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ebfd3eb2-f81e-4bd0-879a-9dc41e281058 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '414928a69dab4a61893cf799bc3cf1a8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:39.501034 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1197/2422] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:41:39 2022] DELETE /identity/v3/users/414928a69dab4a61893cf799bc3cf1a8 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:41:39.503326 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a7885-21e9-41b4-af1b-b508291fab6e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:39.503608 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a7885-21e9-41b4-af1b-b508291fab6e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:39.503883 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a7885-21e9-41b4-af1b-b508291fab6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:39.529111 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:39.545086 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-797a7885-21e9-41b4-af1b-b508291fab6e None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:41:39.546340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-797a7885-21e9-41b4-af1b-b508291fab6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:39.639905 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1226/2423] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:41:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:41:39.655360 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd3eb2-f81e-4bd0-879a-9dc41e281058 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:39.657280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ebfd3eb2-f81e-4bd0-879a-9dc41e281058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:39.759384 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:39.766404 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd3eb2-f81e-4bd0-879a-9dc41e281058 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:39.778567 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83836e89-5eb9-4275-bc7e-435415d11435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:39.780389 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83836e89-5eb9-4275-bc7e-435415d11435 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:39.784278 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83836e89-5eb9-4275-bc7e-435415d11435 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:39.784278 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83836e89-5eb9-4275-bc7e-435415d11435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:39.794157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83836e89-5eb9-4275-bc7e-435415d11435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:39.799402 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83836e89-5eb9-4275-bc7e-435415d11435 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:39.821492 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1198/2424] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:41:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:41:40.535340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a7885-21e9-41b4-af1b-b508291fab6e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:40.536640 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-797a7885-21e9-41b4-af1b-b508291fab6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:40.618088 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a7885-21e9-41b4-af1b-b508291fab6e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:40.625278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045b240-8131-4cf1-a6d0-beae57baf9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:40.629616 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045b240-8131-4cf1-a6d0-beae57baf9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:40.630660 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045b240-8131-4cf1-a6d0-beae57baf9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:40.631475 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045b240-8131-4cf1-a6d0-beae57baf9b3 admin admin] PATH_INFO: `/v3/projects/5b777747392942a2b7ec1b6aa7d7bafa` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:40.633740 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045b240-8131-4cf1-a6d0-beae57baf9b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b777747392942a2b7ec1b6aa7d7bafa)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:40.636507 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045b240-8131-4cf1-a6d0-beae57baf9b3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:40.807398 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7045b240-8131-4cf1-a6d0-beae57baf9b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b777747392942a2b7ec1b6aa7d7bafa', 'request_id': 'req-7045b240-8131-4cf1-a6d0-beae57baf9b3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:40.816578 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7045b240-8131-4cf1-a6d0-beae57baf9b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b777747392942a2b7ec1b6aa7d7bafa', 'request_id': 'req-7045b240-8131-4cf1-a6d0-beae57baf9b3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:40.892079 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7045b240-8131-4cf1-a6d0-beae57baf9b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b777747392942a2b7ec1b6aa7d7bafa', 'request_id': 'req-7045b240-8131-4cf1-a6d0-beae57baf9b3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:40.908531 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7045b240-8131-4cf1-a6d0-beae57baf9b3 admin admin] The token cache is being invalidate because project 5b777747392942a2b7ec1b6aa7d7bafa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:41:40.909192 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7045b240-8131-4cf1-a6d0-beae57baf9b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:40.909907 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7045b240-8131-4cf1-a6d0-beae57baf9b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:40.911639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1227/2425] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:41:40 2022] DELETE /identity/v3/projects/5b777747392942a2b7ec1b6aa7d7bafa => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:41:40.917576 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340704ef-6671-4a5e-a7d3-1244538e6181 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:40.918160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340704ef-6671-4a5e-a7d3-1244538e6181 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:40.918274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340704ef-6671-4a5e-a7d3-1244538e6181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:40.971492 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-340704ef-6671-4a5e-a7d3-1244538e6181 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:41:40.973784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-340704ef-6671-4a5e-a7d3-1244538e6181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:41.093172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1199/2426] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:41:41.213647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f1bad-86eb-47db-accc-3c705669e212 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:41.214139 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f1bad-86eb-47db-accc-3c705669e212 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:41.214624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f1bad-86eb-47db-accc-3c705669e212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:41.252576 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-392f1bad-86eb-47db-accc-3c705669e212 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:41:41.253945 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-392f1bad-86eb-47db-accc-3c705669e212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:41.267703 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340704ef-6671-4a5e-a7d3-1244538e6181 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:41.276989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1228/2427] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:41:41.282717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340704ef-6671-4a5e-a7d3-1244538e6181 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:41.289534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6f706-fccf-4f37-9764-f547015e8c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:41.290417 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6f706-fccf-4f37-9764-f547015e8c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:41.290704 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6f706-fccf-4f37-9764-f547015e8c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:41.291045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6f706-fccf-4f37-9764-f547015e8c20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:41.291853 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6f706-fccf-4f37-9764-f547015e8c20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:41:41.292230 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6f706-fccf-4f37-9764-f547015e8c20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:41.294043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6f706-fccf-4f37-9764-f547015e8c20 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:41.318860 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1200/2428] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:41:41.321289 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f1bad-86eb-47db-accc-3c705669e212 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:41.334668 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f1bad-86eb-47db-accc-3c705669e212 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:41.342182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06d793-53e5-4b58-abb3-d4f65b2ce2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:41.343081 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06d793-53e5-4b58-abb3-d4f65b2ce2e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:41.343457 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06d793-53e5-4b58-abb3-d4f65b2ce2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:41.343839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06d793-53e5-4b58-abb3-d4f65b2ce2e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:41.344767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06d793-53e5-4b58-abb3-d4f65b2ce2e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:41.353245 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06d793-53e5-4b58-abb3-d4f65b2ce2e3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:41.462571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1229/2429] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:41:41 2022] POST /identity/v3/projects => generated 397 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:41:41.465738 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6f706-fccf-4f37-9764-f547015e8c20 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:41.484340 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6f706-fccf-4f37-9764-f547015e8c20 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:41.574185 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16795657-d6c1-43fd-ad59-f6c46fe632ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:41.576299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16795657-d6c1-43fd-ad59-f6c46fe632ac admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:41.576949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16795657-d6c1-43fd-ad59-f6c46fe632ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:41.577450 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16795657-d6c1-43fd-ad59-f6c46fe632ac admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:41.578470 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16795657-d6c1-43fd-ad59-f6c46fe632ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:41.580632 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16795657-d6c1-43fd-ad59-f6c46fe632ac admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:41.682429 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06d793-53e5-4b58-abb3-d4f65b2ce2e3 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:41.683590 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1201/2430] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:41:41 2022] POST /identity/v3/users => generated 299 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:41:41.696656 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06d793-53e5-4b58-abb3-d4f65b2ce2e3 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:41.700959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a10f1-8624-4958-b49a-913b2c0b38b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:41.702337 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a10f1-8624-4958-b49a-913b2c0b38b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:41.702629 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a10f1-8624-4958-b49a-913b2c0b38b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:41.704943 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a10f1-8624-4958-b49a-913b2c0b38b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:41.704943 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a10f1-8624-4958-b49a-913b2c0b38b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:41.705517 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a10f1-8624-4958-b49a-913b2c0b38b7 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:41.717777 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1230/2431] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:41:41.721700 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16795657-d6c1-43fd-ad59-f6c46fe632ac admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:41.741122 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16795657-d6c1-43fd-ad59-f6c46fe632ac admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:41.747640 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076b300-d9af-4f96-b4c2-c2c29c035949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:41.748961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076b300-d9af-4f96-b4c2-c2c29c035949 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:41.748961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076b300-d9af-4f96-b4c2-c2c29c035949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:41.749386 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076b300-d9af-4f96-b4c2-c2c29c035949 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:41.750456 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076b300-d9af-4f96-b4c2-c2c29c035949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:41.752197 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076b300-d9af-4f96-b4c2-c2c29c035949 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:41.768626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1202/2432] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:41:41.773703 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a10f1-8624-4958-b49a-913b2c0b38b7 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:41.789905 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a10f1-8624-4958-b49a-913b2c0b38b7 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:41.795506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed168877-01dc-4282-a5fa-8afb1b846899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:41.796409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed168877-01dc-4282-a5fa-8afb1b846899 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:41.796693 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed168877-01dc-4282-a5fa-8afb1b846899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:41.796969 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed168877-01dc-4282-a5fa-8afb1b846899 admin admin] PATH_INFO: `/v3/projects/7ac84b7bb08942b593d42d2a418e94d7/users/79f64ac318e549929072daf2f7704819/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:41.813941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed168877-01dc-4282-a5fa-8afb1b846899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac84b7bb08942b593d42d2a418e94d7, user_id=79f64ac318e549929072daf2f7704819, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:41.816618 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed168877-01dc-4282-a5fa-8afb1b846899 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:41.927427 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1231/2433] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:41:41 2022] PUT /identity/v3/projects/7ac84b7bb08942b593d42d2a418e94d7/users/79f64ac318e549929072daf2f7704819/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:41:41.937962 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab3810-4b05-44da-b767-dfecd404a0e1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:41.938765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab3810-4b05-44da-b767-dfecd404a0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:41.938765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab3810-4b05-44da-b767-dfecd404a0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:41.969285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5aab3810-4b05-44da-b767-dfecd404a0e1 None None] MFA Rules not processed for user `79f64ac318e549929072daf2f7704819`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:41:41.971153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5aab3810-4b05-44da-b767-dfecd404a0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:42.030987 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:42.054577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1203/2434] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:41:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:41:42.068690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed168877-01dc-4282-a5fa-8afb1b846899 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:42.078096 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ed168877-01dc-4282-a5fa-8afb1b846899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:42.117953 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:42.145497 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed168877-01dc-4282-a5fa-8afb1b846899 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:42.150049 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c673f2-ba45-4984-aa61-684848720182 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:42.150704 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c673f2-ba45-4984-aa61-684848720182 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:42.150983 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c673f2-ba45-4984-aa61-684848720182 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:42.151316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c673f2-ba45-4984-aa61-684848720182 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:42.160756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c673f2-ba45-4984-aa61-684848720182 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:42.162364 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c673f2-ba45-4984-aa61-684848720182 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:42.173147 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1232/2435] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:41:42 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:41:52.102745 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:52.200513 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=14 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:52.583908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab3810-4b05-44da-b767-dfecd404a0e1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:52.648769 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab3810-4b05-44da-b767-dfecd404a0e1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:52.653427 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edff9c-421b-49ce-b725-2e9904597a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:52.654400 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edff9c-421b-49ce-b725-2e9904597a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:52.654581 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edff9c-421b-49ce-b725-2e9904597a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:52.654853 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edff9c-421b-49ce-b725-2e9904597a3b admin admin] PATH_INFO: `/v3/users/79f64ac318e549929072daf2f7704819` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:52.656361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6edff9c-421b-49ce-b725-2e9904597a3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=79f64ac318e549929072daf2f7704819)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:52.658618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6edff9c-421b-49ce-b725-2e9904597a3b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:52.831510 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a6edff9c-421b-49ce-b725-2e9904597a3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79f64ac318e549929072daf2f7704819'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:52.832290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a6edff9c-421b-49ce-b725-2e9904597a3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79f64ac318e549929072daf2f7704819'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:52.842634 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a6edff9c-421b-49ce-b725-2e9904597a3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79f64ac318e549929072daf2f7704819'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:52.843544 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a6edff9c-421b-49ce-b725-2e9904597a3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79f64ac318e549929072daf2f7704819'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:52.858656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a6edff9c-421b-49ce-b725-2e9904597a3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79f64ac318e549929072daf2f7704819'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:52.954277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1204/2436] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:41:52 2022] DELETE /identity/v3/users/79f64ac318e549929072daf2f7704819 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:41:52.959198 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55142cf-fd2d-4ffb-8dad-59a295e2b357 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:52.963198 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55142cf-fd2d-4ffb-8dad-59a295e2b357 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:52.963198 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55142cf-fd2d-4ffb-8dad-59a295e2b357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:52.979435 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c55142cf-fd2d-4ffb-8dad-59a295e2b357 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:41:52.981293 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c55142cf-fd2d-4ffb-8dad-59a295e2b357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:53.056557 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1233/2437] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:41:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:41:53.067114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edff9c-421b-49ce-b725-2e9904597a3b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:53.068947 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a6edff9c-421b-49ce-b725-2e9904597a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:53.145409 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edff9c-421b-49ce-b725-2e9904597a3b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:53.150453 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75cf6ab-22e1-4b4a-8aa1-2779891be59b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:53.151371 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75cf6ab-22e1-4b4a-8aa1-2779891be59b service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:53.151847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75cf6ab-22e1-4b4a-8aa1-2779891be59b service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:53.152324 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75cf6ab-22e1-4b4a-8aa1-2779891be59b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:53.165510 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75cf6ab-22e1-4b4a-8aa1-2779891be59b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:53.167014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75cf6ab-22e1-4b4a-8aa1-2779891be59b service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:53.184989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1205/2438] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:41:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:41:53.719764 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55142cf-fd2d-4ffb-8dad-59a295e2b357 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:53.721578 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c55142cf-fd2d-4ffb-8dad-59a295e2b357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:53.734526 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55142cf-fd2d-4ffb-8dad-59a295e2b357 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:53.739816 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ba47c-a069-4d70-9a41-442c7a85841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:53.741738 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ba47c-a069-4d70-9a41-442c7a85841d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:53.741738 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ba47c-a069-4d70-9a41-442c7a85841d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:53.742121 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ba47c-a069-4d70-9a41-442c7a85841d admin admin] PATH_INFO: `/v3/projects/7ac84b7bb08942b593d42d2a418e94d7` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:53.743187 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ba47c-a069-4d70-9a41-442c7a85841d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ac84b7bb08942b593d42d2a418e94d7)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:53.745198 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ba47c-a069-4d70-9a41-442c7a85841d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:53.891906 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-806ba47c-a069-4d70-9a41-442c7a85841d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ac84b7bb08942b593d42d2a418e94d7', 'request_id': 'req-806ba47c-a069-4d70-9a41-442c7a85841d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:53.899016 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-806ba47c-a069-4d70-9a41-442c7a85841d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ac84b7bb08942b593d42d2a418e94d7', 'request_id': 'req-806ba47c-a069-4d70-9a41-442c7a85841d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:53.970665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-806ba47c-a069-4d70-9a41-442c7a85841d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ac84b7bb08942b593d42d2a418e94d7', 'request_id': 'req-806ba47c-a069-4d70-9a41-442c7a85841d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:53.982315 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-806ba47c-a069-4d70-9a41-442c7a85841d admin admin] The token cache is being invalidate because project 7ac84b7bb08942b593d42d2a418e94d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:41:53.982598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-806ba47c-a069-4d70-9a41-442c7a85841d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:53.983257 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-806ba47c-a069-4d70-9a41-442c7a85841d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:41:53.984864 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1234/2439] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:41:53 2022] DELETE /identity/v3/projects/7ac84b7bb08942b593d42d2a418e94d7 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:41:57.818370 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0738917-66f5-49b6-96a2-4dae5e1cda94 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:57.818890 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0738917-66f5-49b6-96a2-4dae5e1cda94 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:57.818989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0738917-66f5-49b6-96a2-4dae5e1cda94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:57.839026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b0738917-66f5-49b6-96a2-4dae5e1cda94 None None] MFA Rules not processed for user `bbf0ce396885472bb5d225aa009043f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:41:57.840244 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b0738917-66f5-49b6-96a2-4dae5e1cda94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:57.858877 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:41:57.897623 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1206/2440] 10.222.0.17 () {58 vars in 967 bytes} [Tue Jun 7 08:41:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:41:57.910818 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ba47c-a069-4d70-9a41-442c7a85841d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:41:57.912617 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-806ba47c-a069-4d70-9a41-442c7a85841d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:41:57.985991 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ba47c-a069-4d70-9a41-442c7a85841d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:41:57.991323 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71087ffe-6d76-4f73-82e0-af9aeb28214f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334dc47ae2e643ccb44d25c6b5906f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:41:57.991968 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71087ffe-6d76-4f73-82e0-af9aeb28214f service swift] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:41:57.994609 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71087ffe-6d76-4f73-82e0-af9aeb28214f service swift] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:41:57.994609 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71087ffe-6d76-4f73-82e0-af9aeb28214f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:41:58.001584 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71087ffe-6d76-4f73-82e0-af9aeb28214f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:41:58.003206 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71087ffe-6d76-4f73-82e0-af9aeb28214f service swift] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:41:58.012388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1235/2441] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jun 7 08:41:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:42:08.008972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:42:08.028655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=15 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:42:11.704502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0738917-66f5-49b6-96a2-4dae5e1cda94 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:42:11.705955 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b0738917-66f5-49b6-96a2-4dae5e1cda94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:42:11.766190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0738917-66f5-49b6-96a2-4dae5e1cda94 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:42:11.769945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0973e-8f39-42b6-9f12-dd92697c85be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:42:11.770602 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0973e-8f39-42b6-9f12-dd92697c85be service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:42:11.770884 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0973e-8f39-42b6-9f12-dd92697c85be service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:42:11.771160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0973e-8f39-42b6-9f12-dd92697c85be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:42:11.773043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3fb0973e-8f39-42b6-9f12-dd92697c85be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:42:11.849544 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb0973e-8f39-42b6-9f12-dd92697c85be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:42:11.851162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb0973e-8f39-42b6-9f12-dd92697c85be service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:42:11.861442 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1207/2442] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:42:11 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:42:21.856418 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=50 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:42:27.309462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d415ea6-80a4-46bb-9a76-782ef62837bc None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:42:27.310012 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d415ea6-80a4-46bb-9a76-782ef62837bc None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:42:27.310012 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d415ea6-80a4-46bb-9a76-782ef62837bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:42:27.328620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-5d415ea6-80a4-46bb-9a76-782ef62837bc None None] MFA Rules not processed for user `055ec8a54bdf400fac0a46b0e5497f1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:42:27.329858 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5d415ea6-80a4-46bb-9a76-782ef62837bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:42:27.397486 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1236/2443] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:42:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:42:37.390052 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=25 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:42:50.958185 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f9361-d5a8-4ac9-88b9-29b728e4ee62 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:42:50.969975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f9361-d5a8-4ac9-88b9-29b728e4ee62 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:42:50.969975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f9361-d5a8-4ac9-88b9-29b728e4ee62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:42:50.978228 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-248f9361-d5a8-4ac9-88b9-29b728e4ee62 None None] MFA Rules not processed for user `0b2efb9857cf495f8b51a96b5f868d44`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:42:50.979427 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-248f9361-d5a8-4ac9-88b9-29b728e4ee62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:42:51.035812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1208/2444] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:42:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:43:01.028859 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=25 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:43:11.079294 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d415ea6-80a4-46bb-9a76-782ef62837bc None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:11.080774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5d415ea6-80a4-46bb-9a76-782ef62837bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:11.095074 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d415ea6-80a4-46bb-9a76-782ef62837bc None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:11.101830 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784cba5-1b5a-4e15-83f1-110a510e58d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:11.102622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784cba5-1b5a-4e15-83f1-110a510e58d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:11.103031 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784cba5-1b5a-4e15-83f1-110a510e58d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:11.103514 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784cba5-1b5a-4e15-83f1-110a510e58d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:11.106330 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f784cba5-1b5a-4e15-83f1-110a510e58d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:11.207248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784cba5-1b5a-4e15-83f1-110a510e58d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:11.209642 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784cba5-1b5a-4e15-83f1-110a510e58d0 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:11.224058 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1237/2445] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:43:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:43:14.485801 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f9361-d5a8-4ac9-88b9-29b728e4ee62 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:14.495864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f9361-d5a8-4ac9-88b9-29b728e4ee62 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:14.500334 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7443a6-ffdb-48cc-befc-01736d45ef37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:14.500983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7443a6-ffdb-48cc-befc-01736d45ef37 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:14.501301 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7443a6-ffdb-48cc-befc-01736d45ef37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:14.501936 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7443a6-ffdb-48cc-befc-01736d45ef37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:14.504062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cf7443a6-ffdb-48cc-befc-01736d45ef37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:14.519209 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7443a6-ffdb-48cc-befc-01736d45ef37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:14.520662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7443a6-ffdb-48cc-befc-01736d45ef37 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:14.533292 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1209/2446] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:43:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:43:17.598807 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947f1c6-0f00-4a56-b2be-3f1b55ae1791 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:17.599371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947f1c6-0f00-4a56-b2be-3f1b55ae1791 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:17.599478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947f1c6-0f00-4a56-b2be-3f1b55ae1791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:17.617952 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-2947f1c6-0f00-4a56-b2be-3f1b55ae1791 None None] MFA Rules not processed for user `4e5893f0909446ff82b5a72584ecae37`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:43:17.619161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2947f1c6-0f00-4a56-b2be-3f1b55ae1791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:17.674658 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1238/2447] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:43:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:43:18.553371 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7443a6-ffdb-48cc-befc-01736d45ef37 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:18.555543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cf7443a6-ffdb-48cc-befc-01736d45ef37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:18.652322 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7443a6-ffdb-48cc-befc-01736d45ef37 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:18.665311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2e56a-34d8-45e5-a76e-e227ab9a3da8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc0d7bd7e4547f4a8d90ffc2c2e1d03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:18.666225 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2e56a-34d8-45e5-a76e-e227ab9a3da8 service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:18.666672 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2e56a-34d8-45e5-a76e-e227ab9a3da8 service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:18.667124 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2e56a-34d8-45e5-a76e-e227ab9a3da8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:18.669219 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-78f2e56a-34d8-45e5-a76e-e227ab9a3da8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:18.681773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2e56a-34d8-45e5-a76e-e227ab9a3da8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:18.683342 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2e56a-34d8-45e5-a76e-e227ab9a3da8 service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:18.694864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1210/2448] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:43:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:43:19.358953 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947f1c6-0f00-4a56-b2be-3f1b55ae1791 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:19.360890 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2947f1c6-0f00-4a56-b2be-3f1b55ae1791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:19.382485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947f1c6-0f00-4a56-b2be-3f1b55ae1791 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:19.388457 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de5d12-61f9-4190-927d-caa9ed1ab082 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:19.396299 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de5d12-61f9-4190-927d-caa9ed1ab082 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:19.396299 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de5d12-61f9-4190-927d-caa9ed1ab082 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:19.396299 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de5d12-61f9-4190-927d-caa9ed1ab082 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:19.396299 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-21de5d12-61f9-4190-927d-caa9ed1ab082 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:19.414434 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de5d12-61f9-4190-927d-caa9ed1ab082 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:19.416198 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de5d12-61f9-4190-927d-caa9ed1ab082 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:19.427773 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1239/2449] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:43:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:43:28.472458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2e56a-34d8-45e5-a76e-e227ab9a3da8 service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:28.481745 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2e56a-34d8-45e5-a76e-e227ab9a3da8 service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:28.485577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a5f92-8b16-4306-8a2f-d84ba4719944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:28.486221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a5f92-8b16-4306-8a2f-d84ba4719944 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:28.486501 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a5f92-8b16-4306-8a2f-d84ba4719944 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:28.486773 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a5f92-8b16-4306-8a2f-d84ba4719944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:28.488604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5d4a5f92-8b16-4306-8a2f-d84ba4719944 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:28.499118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a5f92-8b16-4306-8a2f-d84ba4719944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:28.500474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a5f92-8b16-4306-8a2f-d84ba4719944 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:28.508489 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1211/2450] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:43:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:43:29.422727 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=65 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:43:35.627566 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de5d12-61f9-4190-927d-caa9ed1ab082 service nova] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:35.639008 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de5d12-61f9-4190-927d-caa9ed1ab082 service nova] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:35.644644 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff0f46-1eca-4bd8-b428-2ad16ed19404 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:35.645864 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff0f46-1eca-4bd8-b428-2ad16ed19404 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:35.646160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff0f46-1eca-4bd8-b428-2ad16ed19404 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:35.646468 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff0f46-1eca-4bd8-b428-2ad16ed19404 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:35.648453 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-beff0f46-1eca-4bd8-b428-2ad16ed19404 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:35.661567 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff0f46-1eca-4bd8-b428-2ad16ed19404 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:35.663419 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff0f46-1eca-4bd8-b428-2ad16ed19404 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:35.688455 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1240/2451] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:43:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:43:35.766383 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a5f92-8b16-4306-8a2f-d84ba4719944 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:35.787299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a5f92-8b16-4306-8a2f-d84ba4719944 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:35.804228 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d7aa3-f70f-47ca-b2a4-021ff99d4c62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:35.805056 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d7aa3-f70f-47ca-b2a4-021ff99d4c62 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:35.806553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d7aa3-f70f-47ca-b2a4-021ff99d4c62 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:35.807606 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d7aa3-f70f-47ca-b2a4-021ff99d4c62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:35.813150 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d31d7aa3-f70f-47ca-b2a4-021ff99d4c62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:35.843860 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d7aa3-f70f-47ca-b2a4-021ff99d4c62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:35.846426 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d7aa3-f70f-47ca-b2a4-021ff99d4c62 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:35.865635 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1212/2452] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:43:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:43:36.505173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff0f46-1eca-4bd8-b428-2ad16ed19404 service nova] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:36.516191 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff0f46-1eca-4bd8-b428-2ad16ed19404 service nova] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:36.519903 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7413b2-5b7b-423c-ad64-49a37d61635f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:36.520557 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7413b2-5b7b-423c-ad64-49a37d61635f service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:36.520836 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7413b2-5b7b-423c-ad64-49a37d61635f service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:36.521307 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7413b2-5b7b-423c-ad64-49a37d61635f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:36.523831 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3c7413b2-5b7b-423c-ad64-49a37d61635f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:36.603030 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7413b2-5b7b-423c-ad64-49a37d61635f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:36.604798 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7413b2-5b7b-423c-ad64-49a37d61635f service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:36.617178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1241/2453] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:43:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:43:40.486992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d7aa3-f70f-47ca-b2a4-021ff99d4c62 service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:40.496222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d31d7aa3-f70f-47ca-b2a4-021ff99d4c62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:40.508105 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d7aa3-f70f-47ca-b2a4-021ff99d4c62 service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:40.516436 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01949135-18f4-48bc-ad6e-d0cf5aebe4f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc0d7bd7e4547f4a8d90ffc2c2e1d03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:40.517631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01949135-18f4-48bc-ad6e-d0cf5aebe4f4 service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:40.517928 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01949135-18f4-48bc-ad6e-d0cf5aebe4f4 service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:40.518211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01949135-18f4-48bc-ad6e-d0cf5aebe4f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:40.521698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-01949135-18f4-48bc-ad6e-d0cf5aebe4f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:40.545220 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01949135-18f4-48bc-ad6e-d0cf5aebe4f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:40.546894 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01949135-18f4-48bc-ad6e-d0cf5aebe4f4 service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:40.557950 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1213/2454] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:43:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:43:40.632563 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7413b2-5b7b-423c-ad64-49a37d61635f service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:40.634143 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3c7413b2-5b7b-423c-ad64-49a37d61635f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:40.647316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7413b2-5b7b-423c-ad64-49a37d61635f service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:40.652664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32608b43-3f8d-48b3-b835-e6ab93fba127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:40.653587 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32608b43-3f8d-48b3-b835-e6ab93fba127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:40.653912 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32608b43-3f8d-48b3-b835-e6ab93fba127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:40.654193 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32608b43-3f8d-48b3-b835-e6ab93fba127 admin admin] PATH_INFO: `/v3/users/055ec8a54bdf400fac0a46b0e5497f1b` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:40.655632 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32608b43-3f8d-48b3-b835-e6ab93fba127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=055ec8a54bdf400fac0a46b0e5497f1b)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:40.657614 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32608b43-3f8d-48b3-b835-e6ab93fba127 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:40.827952 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-32608b43-3f8d-48b3-b835-e6ab93fba127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '055ec8a54bdf400fac0a46b0e5497f1b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:40.828538 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-32608b43-3f8d-48b3-b835-e6ab93fba127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '055ec8a54bdf400fac0a46b0e5497f1b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:40.835395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-32608b43-3f8d-48b3-b835-e6ab93fba127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '055ec8a54bdf400fac0a46b0e5497f1b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:40.836028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-32608b43-3f8d-48b3-b835-e6ab93fba127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '055ec8a54bdf400fac0a46b0e5497f1b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:40.905695 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-32608b43-3f8d-48b3-b835-e6ab93fba127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '055ec8a54bdf400fac0a46b0e5497f1b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:40.923397 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1242/2455] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:43:40 2022] DELETE /identity/v3/users/055ec8a54bdf400fac0a46b0e5497f1b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:43:41.187937 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01949135-18f4-48bc-ad6e-d0cf5aebe4f4 service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:41.194868 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=57 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:43:41.204647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01949135-18f4-48bc-ad6e-d0cf5aebe4f4 service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:41.263123 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee9273-ef7b-40ba-9425-4fbd34627090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:41.264003 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee9273-ef7b-40ba-9425-4fbd34627090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:41.264280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee9273-ef7b-40ba-9425-4fbd34627090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:41.264553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee9273-ef7b-40ba-9425-4fbd34627090 admin admin] PATH_INFO: `/v3/projects/10b79d91933b4f7f9f32d66138e18373` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:41.265743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee9273-ef7b-40ba-9425-4fbd34627090 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10b79d91933b4f7f9f32d66138e18373)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:41.267632 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee9273-ef7b-40ba-9425-4fbd34627090 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:41.364341 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-44ee9273-ef7b-40ba-9425-4fbd34627090 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10b79d91933b4f7f9f32d66138e18373', 'request_id': 'req-44ee9273-ef7b-40ba-9425-4fbd34627090'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:41.371700 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-44ee9273-ef7b-40ba-9425-4fbd34627090 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10b79d91933b4f7f9f32d66138e18373', 'request_id': 'req-44ee9273-ef7b-40ba-9425-4fbd34627090'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:41.378563 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-44ee9273-ef7b-40ba-9425-4fbd34627090 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10b79d91933b4f7f9f32d66138e18373', 'request_id': 'req-44ee9273-ef7b-40ba-9425-4fbd34627090'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:41.492854 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-44ee9273-ef7b-40ba-9425-4fbd34627090 admin admin] The token cache is being invalidate because project 10b79d91933b4f7f9f32d66138e18373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:43:41.493534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-44ee9273-ef7b-40ba-9425-4fbd34627090 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:41.494026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-44ee9273-ef7b-40ba-9425-4fbd34627090 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:41.496296 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1214/2456] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:43:41 2022] DELETE /identity/v3/projects/10b79d91933b4f7f9f32d66138e18373 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:43:41.507610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971851f4-500c-435c-8ed9-02f0aeb9c382 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:41.507899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971851f4-500c-435c-8ed9-02f0aeb9c382 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:41.508176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971851f4-500c-435c-8ed9-02f0aeb9c382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:41.526425 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-971851f4-500c-435c-8ed9-02f0aeb9c382 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:43:41.535223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-971851f4-500c-435c-8ed9-02f0aeb9c382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:41.575897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:43:41.588516 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1243/2457] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:43:41.596732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615dcfd-4df3-4632-9572-1deb1520253e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:41.597114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615dcfd-4df3-4632-9572-1deb1520253e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:41.597358 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615dcfd-4df3-4632-9572-1deb1520253e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:41.616638 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7615dcfd-4df3-4632-9572-1deb1520253e None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:43:41.618053 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7615dcfd-4df3-4632-9572-1deb1520253e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:41.627659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1215/2458] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:43:41.631826 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971851f4-500c-435c-8ed9-02f0aeb9c382 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:41.641781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971851f4-500c-435c-8ed9-02f0aeb9c382 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:41.646358 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6d467-8960-46b1-9056-d456aa463a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:41.647009 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6d467-8960-46b1-9056-d456aa463a3c admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:41.647284 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6d467-8960-46b1-9056-d456aa463a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:41.647555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6d467-8960-46b1-9056-d456aa463a3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:41.648272 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6d467-8960-46b1-9056-d456aa463a3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:43:41.648576 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6d467-8960-46b1-9056-d456aa463a3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:41.650202 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6d467-8960-46b1-9056-d456aa463a3c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:41.662192 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1244/2459] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:43:41.667054 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615dcfd-4df3-4632-9572-1deb1520253e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:41.676223 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615dcfd-4df3-4632-9572-1deb1520253e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:41.680987 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500074a-c580-47da-83da-1f12bba9beab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:41.681692 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500074a-c580-47da-83da-1f12bba9beab admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:41.682489 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500074a-c580-47da-83da-1f12bba9beab admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:41.682764 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500074a-c580-47da-83da-1f12bba9beab admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:41.683456 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500074a-c580-47da-83da-1f12bba9beab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:41.685382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500074a-c580-47da-83da-1f12bba9beab admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:41.747487 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1216/2460] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:43:41 2022] POST /identity/v3/projects => generated 377 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:43:41.751630 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6d467-8960-46b1-9056-d456aa463a3c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:41.762171 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6d467-8960-46b1-9056-d456aa463a3c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:41.818977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7724c5-47a0-4c12-8cab-ac88540bdd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:41.819629 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7724c5-47a0-4c12-8cab-ac88540bdd51 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:41.819912 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7724c5-47a0-4c12-8cab-ac88540bdd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:41.820180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7724c5-47a0-4c12-8cab-ac88540bdd51 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:41.820857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7724c5-47a0-4c12-8cab-ac88540bdd51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:41.822812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7724c5-47a0-4c12-8cab-ac88540bdd51 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:41.882912 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1245/2461] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:43:41 2022] POST /identity/v3/users => generated 289 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:43:41.887085 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500074a-c580-47da-83da-1f12bba9beab admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:41.897538 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500074a-c580-47da-83da-1f12bba9beab admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:41.902626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92005bf7-6be4-4913-a2ab-ae908c30d9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:41.903281 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92005bf7-6be4-4913-a2ab-ae908c30d9a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:41.903566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92005bf7-6be4-4913-a2ab-ae908c30d9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:41.903845 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92005bf7-6be4-4913-a2ab-ae908c30d9a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:41.904565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92005bf7-6be4-4913-a2ab-ae908c30d9a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:41.906515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92005bf7-6be4-4913-a2ab-ae908c30d9a5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:41.916953 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1217/2462] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:43:41.927263 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7724c5-47a0-4c12-8cab-ac88540bdd51 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:41.937247 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7724c5-47a0-4c12-8cab-ac88540bdd51 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:41.941970 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cac424-21b6-4498-b3a8-12be0d1e6faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:41.942617 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cac424-21b6-4498-b3a8-12be0d1e6faf admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:41.942902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cac424-21b6-4498-b3a8-12be0d1e6faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:41.943181 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cac424-21b6-4498-b3a8-12be0d1e6faf admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:41.943906 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cac424-21b6-4498-b3a8-12be0d1e6faf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:41.945547 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cac424-21b6-4498-b3a8-12be0d1e6faf admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:41.955386 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1246/2463] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:43:41.960053 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92005bf7-6be4-4913-a2ab-ae908c30d9a5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:41.970409 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92005bf7-6be4-4913-a2ab-ae908c30d9a5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:41.975549 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd9ccd-759b-4e1b-92dc-d231f08d3035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:41.976446 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd9ccd-759b-4e1b-92dc-d231f08d3035 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:41.976742 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd9ccd-759b-4e1b-92dc-d231f08d3035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:41.977045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd9ccd-759b-4e1b-92dc-d231f08d3035 admin admin] PATH_INFO: `/v3/projects/e28f24d8d35f4336be9d43c81bef8b41/users/c856e25bd5a14c618b90889c9f4f65b4/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:41.991876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd9ccd-759b-4e1b-92dc-d231f08d3035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e28f24d8d35f4336be9d43c81bef8b41, user_id=c856e25bd5a14c618b90889c9f4f65b4, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:41.994592 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd9ccd-759b-4e1b-92dc-d231f08d3035 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:42.072455 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1218/2464] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:43:41 2022] PUT /identity/v3/projects/e28f24d8d35f4336be9d43c81bef8b41/users/c856e25bd5a14c618b90889c9f4f65b4/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:43:42.083235 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a2aaf-d408-41bd-a794-4f8bc4e6ee06 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:42.083616 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a2aaf-d408-41bd-a794-4f8bc4e6ee06 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:42.083895 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a2aaf-d408-41bd-a794-4f8bc4e6ee06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:42.112833 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-275a2aaf-d408-41bd-a794-4f8bc4e6ee06 None None] MFA Rules not processed for user `c856e25bd5a14c618b90889c9f4f65b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:43:42.114269 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-275a2aaf-d408-41bd-a794-4f8bc4e6ee06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:42.192076 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1247/2465] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:43:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:43:42.196718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd9ccd-759b-4e1b-92dc-d231f08d3035 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:42.230268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:43:42.264412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd9ccd-759b-4e1b-92dc-d231f08d3035 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:42.269807 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d983f0-1d05-4791-b79e-b3ca4e655cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:42.270461 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d983f0-1d05-4791-b79e-b3ca4e655cb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:42.270778 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d983f0-1d05-4791-b79e-b3ca4e655cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:42.270998 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d983f0-1d05-4791-b79e-b3ca4e655cb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:42.271745 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d983f0-1d05-4791-b79e-b3ca4e655cb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:42.273701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d983f0-1d05-4791-b79e-b3ca4e655cb4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:42.389613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1219/2466] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:43:42 2022] POST /identity/v3/projects => generated 377 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:43:42.393247 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a2aaf-d408-41bd-a794-4f8bc4e6ee06 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:42.405137 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a2aaf-d408-41bd-a794-4f8bc4e6ee06 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:42.450881 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:43:42.462788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee5ee2-8a92-421d-af6f-9ac38373cef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:42.463454 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee5ee2-8a92-421d-af6f-9ac38373cef0 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:42.463743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee5ee2-8a92-421d-af6f-9ac38373cef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:42.464631 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee5ee2-8a92-421d-af6f-9ac38373cef0 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:42.465370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee5ee2-8a92-421d-af6f-9ac38373cef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:42.467423 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee5ee2-8a92-421d-af6f-9ac38373cef0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:42.533505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1248/2467] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:43:42 2022] POST /identity/v3/users => generated 295 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:43:42.537367 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d983f0-1d05-4791-b79e-b3ca4e655cb4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:42.548043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d983f0-1d05-4791-b79e-b3ca4e655cb4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:42.553279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf34e0-dd6b-4bf5-85ea-606aeb023018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:42.554039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf34e0-dd6b-4bf5-85ea-606aeb023018 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:42.555098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf34e0-dd6b-4bf5-85ea-606aeb023018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:42.555098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf34e0-dd6b-4bf5-85ea-606aeb023018 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:42.555598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf34e0-dd6b-4bf5-85ea-606aeb023018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:42.557326 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf34e0-dd6b-4bf5-85ea-606aeb023018 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:42.574695 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1220/2468] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:43:42.578946 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee5ee2-8a92-421d-af6f-9ac38373cef0 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:42.590372 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee5ee2-8a92-421d-af6f-9ac38373cef0 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:42.596791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef0a04-b08d-4074-916b-6f2f3d3fa93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:42.597951 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef0a04-b08d-4074-916b-6f2f3d3fa93f admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:42.598266 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef0a04-b08d-4074-916b-6f2f3d3fa93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:42.598765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef0a04-b08d-4074-916b-6f2f3d3fa93f admin admin] PATH_INFO: `/v3/projects/de670bb8290d488d9abd42634361435c/users/d446a323e1434e32b64bccc28be4b137/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:42.626378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef0a04-b08d-4074-916b-6f2f3d3fa93f admin admin] RBAC: Authorizing `identity:create_grant(project_id=de670bb8290d488d9abd42634361435c, user_id=d446a323e1434e32b64bccc28be4b137, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:42.629683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef0a04-b08d-4074-916b-6f2f3d3fa93f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:42.718918 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1249/2469] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:43:42 2022] PUT /identity/v3/projects/de670bb8290d488d9abd42634361435c/users/d446a323e1434e32b64bccc28be4b137/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:43:42.722995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf34e0-dd6b-4bf5-85ea-606aeb023018 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:42.733673 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf34e0-dd6b-4bf5-85ea-606aeb023018 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:42.790794 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17604133-45e7-458a-b3f4-48a1f385b1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:42.791379 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17604133-45e7-458a-b3f4-48a1f385b1a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:42.791672 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17604133-45e7-458a-b3f4-48a1f385b1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:42.791975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17604133-45e7-458a-b3f4-48a1f385b1a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:42.792845 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17604133-45e7-458a-b3f4-48a1f385b1a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:42.794513 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17604133-45e7-458a-b3f4-48a1f385b1a4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:42.804632 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1221/2470] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:43:42.809335 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef0a04-b08d-4074-916b-6f2f3d3fa93f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:42.820044 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef0a04-b08d-4074-916b-6f2f3d3fa93f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:42.825550 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cca5f-1418-4923-85b4-1158dc48951f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:42.826593 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cca5f-1418-4923-85b4-1158dc48951f admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:42.826739 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cca5f-1418-4923-85b4-1158dc48951f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:42.827016 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cca5f-1418-4923-85b4-1158dc48951f admin admin] PATH_INFO: `/v3/projects/de670bb8290d488d9abd42634361435c/users/d446a323e1434e32b64bccc28be4b137/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:42.829062 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343cca5f-1418-4923-85b4-1158dc48951f admin admin] RBAC: Authorizing `identity:create_grant(project_id=de670bb8290d488d9abd42634361435c, user_id=d446a323e1434e32b64bccc28be4b137, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:42.831761 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343cca5f-1418-4923-85b4-1158dc48951f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:42.843195 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1250/2471] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:43:42 2022] PUT /identity/v3/projects/de670bb8290d488d9abd42634361435c/users/d446a323e1434e32b64bccc28be4b137/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:43:42.853253 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d6014-2641-4042-bbf7-5f761cb54435 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:42.853547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d6014-2641-4042-bbf7-5f761cb54435 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:42.860628 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d6014-2641-4042-bbf7-5f761cb54435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:42.883274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6f0d6014-2641-4042-bbf7-5f761cb54435 None None] MFA Rules not processed for user `d446a323e1434e32b64bccc28be4b137`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:43:42.884512 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6f0d6014-2641-4042-bbf7-5f761cb54435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:42.959974 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1222/2472] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:43:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:43:42.970380 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cca5f-1418-4923-85b4-1158dc48951f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:42.972120 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-343cca5f-1418-4923-85b4-1158dc48951f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:43.032626 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cca5f-1418-4923-85b4-1158dc48951f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:43.036319 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94c8a0-9941-41aa-bc10-dd34859b964b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:43.036963 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94c8a0-9941-41aa-bc10-dd34859b964b service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:43.037278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94c8a0-9941-41aa-bc10-dd34859b964b service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:43.037554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94c8a0-9941-41aa-bc10-dd34859b964b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:43.093279 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94c8a0-9941-41aa-bc10-dd34859b964b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:43.094724 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94c8a0-9941-41aa-bc10-dd34859b964b service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:43.103540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1251/2473] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:43:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:43:43.769143 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12465e-2652-4a24-ace5-b30022c2398d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:43.769599 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12465e-2652-4a24-ace5-b30022c2398d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:43.769694 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12465e-2652-4a24-ace5-b30022c2398d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:43.788304 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6a12465e-2652-4a24-ace5-b30022c2398d None None] MFA Rules not processed for user `bbf0ce396885472bb5d225aa009043f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:43:43.789602 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6a12465e-2652-4a24-ace5-b30022c2398d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:43.807889 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:43:43.847724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1223/2474] 10.222.0.17 () {58 vars in 967 bytes} [Tue Jun 7 08:43:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:43:43.956610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94c8a0-9941-41aa-bc10-dd34859b964b service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:43.958057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9a94c8a0-9941-41aa-bc10-dd34859b964b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:44.014926 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:43:44.029950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94c8a0-9941-41aa-bc10-dd34859b964b service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:44.035214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5240d-ed05-4a55-a565-adb40b49ece9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334dc47ae2e643ccb44d25c6b5906f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:44.035862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5240d-ed05-4a55-a565-adb40b49ece9 service swift] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:44.036362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5240d-ed05-4a55-a565-adb40b49ece9 service swift] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:44.036798 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5240d-ed05-4a55-a565-adb40b49ece9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:44.045544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca5240d-ed05-4a55-a565-adb40b49ece9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:44.047241 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca5240d-ed05-4a55-a565-adb40b49ece9 service swift] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:44.055477 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1252/2475] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jun 7 08:43:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:43:49.295749 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12465e-2652-4a24-ace5-b30022c2398d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:49.297240 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6a12465e-2652-4a24-ace5-b30022c2398d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:49.309173 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12465e-2652-4a24-ace5-b30022c2398d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:49.312964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f761a-124e-4d03-b0ab-974c439a3519 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:49.317377 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f761a-124e-4d03-b0ab-974c439a3519 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:49.317377 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f761a-124e-4d03-b0ab-974c439a3519 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:49.317377 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f761a-124e-4d03-b0ab-974c439a3519 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:49.324691 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f761a-124e-4d03-b0ab-974c439a3519 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:49.326158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f761a-124e-4d03-b0ab-974c439a3519 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:49.347068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1224/2476] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:43:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:43:49.903186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5240d-ed05-4a55-a565-adb40b49ece9 service swift] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:49.904734 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-fca5240d-ed05-4a55-a565-adb40b49ece9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:49.995073 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5240d-ed05-4a55-a565-adb40b49ece9 service swift] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:50.014236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77e75b-0a1c-452d-97b9-9920849dee7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:50.014236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77e75b-0a1c-452d-97b9-9920849dee7a service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:50.014236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77e75b-0a1c-452d-97b9-9920849dee7a service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:50.014236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77e75b-0a1c-452d-97b9-9920849dee7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:50.014928 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2d77e75b-0a1c-452d-97b9-9920849dee7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:50.101582 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77e75b-0a1c-452d-97b9-9920849dee7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:50.104037 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77e75b-0a1c-452d-97b9-9920849dee7a service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:50.119810 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1253/2477] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:43:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:43:52.119005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f761a-124e-4d03-b0ab-974c439a3519 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:52.121525 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e69f761a-124e-4d03-b0ab-974c439a3519 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:52.195275 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f761a-124e-4d03-b0ab-974c439a3519 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:52.200122 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74652a3f-87c1-4425-af31-f4b571f3674e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:52.201320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74652a3f-87c1-4425-af31-f4b571f3674e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:52.201320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74652a3f-87c1-4425-af31-f4b571f3674e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:52.201541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74652a3f-87c1-4425-af31-f4b571f3674e admin admin] PATH_INFO: `/v3/users/3bbc17dea01a41ad9ca61bc65ed59cf6` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:52.202748 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74652a3f-87c1-4425-af31-f4b571f3674e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bbc17dea01a41ad9ca61bc65ed59cf6)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:52.204652 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74652a3f-87c1-4425-af31-f4b571f3674e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:52.416138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-74652a3f-87c1-4425-af31-f4b571f3674e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bbc17dea01a41ad9ca61bc65ed59cf6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:52.416803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-74652a3f-87c1-4425-af31-f4b571f3674e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bbc17dea01a41ad9ca61bc65ed59cf6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:52.423791 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-74652a3f-87c1-4425-af31-f4b571f3674e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bbc17dea01a41ad9ca61bc65ed59cf6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:52.424424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-74652a3f-87c1-4425-af31-f4b571f3674e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bbc17dea01a41ad9ca61bc65ed59cf6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:52.436847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-74652a3f-87c1-4425-af31-f4b571f3674e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bbc17dea01a41ad9ca61bc65ed59cf6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:52.500042 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1225/2478] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:43:52 2022] DELETE /identity/v3/users/3bbc17dea01a41ad9ca61bc65ed59cf6 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:43:52.753812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77e75b-0a1c-452d-97b9-9920849dee7a service nova] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:52.766284 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77e75b-0a1c-452d-97b9-9920849dee7a service nova] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:52.827170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d3d1f-89ca-45dc-9a30-c1c24f98c026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:52.828030 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d3d1f-89ca-45dc-9a30-c1c24f98c026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:52.828311 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d3d1f-89ca-45dc-9a30-c1c24f98c026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:52.828586 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d3d1f-89ca-45dc-9a30-c1c24f98c026 admin admin] PATH_INFO: `/v3/projects/1e49af480c6c4449b329b5e71904b17c` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:52.829792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d3d1f-89ca-45dc-9a30-c1c24f98c026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e49af480c6c4449b329b5e71904b17c)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:52.831893 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d3d1f-89ca-45dc-9a30-c1c24f98c026 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:52.856760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:43:52.962163 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c77d3d1f-89ca-45dc-9a30-c1c24f98c026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e49af480c6c4449b329b5e71904b17c', 'request_id': 'req-c77d3d1f-89ca-45dc-9a30-c1c24f98c026'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:52.968846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c77d3d1f-89ca-45dc-9a30-c1c24f98c026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e49af480c6c4449b329b5e71904b17c', 'request_id': 'req-c77d3d1f-89ca-45dc-9a30-c1c24f98c026'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:53.031482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c77d3d1f-89ca-45dc-9a30-c1c24f98c026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e49af480c6c4449b329b5e71904b17c', 'request_id': 'req-c77d3d1f-89ca-45dc-9a30-c1c24f98c026'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:53.042730 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c77d3d1f-89ca-45dc-9a30-c1c24f98c026 admin admin] The token cache is being invalidate because project 1e49af480c6c4449b329b5e71904b17c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:43:53.043337 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c77d3d1f-89ca-45dc-9a30-c1c24f98c026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:53.043639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c77d3d1f-89ca-45dc-9a30-c1c24f98c026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:53.045205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1254/2479] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:43:52 2022] DELETE /identity/v3/projects/1e49af480c6c4449b329b5e71904b17c => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:43:53.049274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74652a3f-87c1-4425-af31-f4b571f3674e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:53.050662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-74652a3f-87c1-4425-af31-f4b571f3674e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:53.063472 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74652a3f-87c1-4425-af31-f4b571f3674e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:53.096508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:43:53.125711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb4621-45b7-4e01-b846-86b8b6be4c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:53.126602 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb4621-45b7-4e01-b846-86b8b6be4c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:53.126885 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb4621-45b7-4e01-b846-86b8b6be4c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:53.127290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb4621-45b7-4e01-b846-86b8b6be4c2e admin admin] PATH_INFO: `/v3/users/11f8e58d1a234cbfa71d799541c70a4c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:53.128566 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb4621-45b7-4e01-b846-86b8b6be4c2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=11f8e58d1a234cbfa71d799541c70a4c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:53.130543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb4621-45b7-4e01-b846-86b8b6be4c2e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:53.313555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-65bb4621-45b7-4e01-b846-86b8b6be4c2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11f8e58d1a234cbfa71d799541c70a4c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:53.314125 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-65bb4621-45b7-4e01-b846-86b8b6be4c2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11f8e58d1a234cbfa71d799541c70a4c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:53.320982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-65bb4621-45b7-4e01-b846-86b8b6be4c2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11f8e58d1a234cbfa71d799541c70a4c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:53.321625 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-65bb4621-45b7-4e01-b846-86b8b6be4c2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11f8e58d1a234cbfa71d799541c70a4c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:53.334101 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-65bb4621-45b7-4e01-b846-86b8b6be4c2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11f8e58d1a234cbfa71d799541c70a4c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:53.425247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1226/2480] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:43:53 2022] DELETE /identity/v3/users/11f8e58d1a234cbfa71d799541c70a4c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:43:53.661141 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d3d1f-89ca-45dc-9a30-c1c24f98c026 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:53.662592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c77d3d1f-89ca-45dc-9a30-c1c24f98c026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:53.678654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d3d1f-89ca-45dc-9a30-c1c24f98c026 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:53.737926 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e513f-7916-4670-ac55-4bdecafd307e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:53.739293 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e513f-7916-4670-ac55-4bdecafd307e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:53.739578 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e513f-7916-4670-ac55-4bdecafd307e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:53.739877 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e513f-7916-4670-ac55-4bdecafd307e admin admin] PATH_INFO: `/v3/projects/74fcd48e72144c8280157e95247c5f47` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:53.741086 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e513f-7916-4670-ac55-4bdecafd307e admin admin] RBAC: Authorizing `identity:delete_project(project_id=74fcd48e72144c8280157e95247c5f47)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:53.742997 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e513f-7916-4670-ac55-4bdecafd307e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:53.818898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b66e513f-7916-4670-ac55-4bdecafd307e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74fcd48e72144c8280157e95247c5f47', 'request_id': 'req-b66e513f-7916-4670-ac55-4bdecafd307e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:53.825750 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b66e513f-7916-4670-ac55-4bdecafd307e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74fcd48e72144c8280157e95247c5f47', 'request_id': 'req-b66e513f-7916-4670-ac55-4bdecafd307e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:53.888341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b66e513f-7916-4670-ac55-4bdecafd307e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74fcd48e72144c8280157e95247c5f47', 'request_id': 'req-b66e513f-7916-4670-ac55-4bdecafd307e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:53.899660 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b66e513f-7916-4670-ac55-4bdecafd307e admin admin] The token cache is being invalidate because project 74fcd48e72144c8280157e95247c5f47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:43:53.901180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b66e513f-7916-4670-ac55-4bdecafd307e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:53.901180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b66e513f-7916-4670-ac55-4bdecafd307e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:43:53.902086 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1255/2481] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:43:53 2022] DELETE /identity/v3/projects/74fcd48e72144c8280157e95247c5f47 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:43:53.907728 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e286c0c-e494-40d2-b063-b95eebe773f1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:53.907966 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e286c0c-e494-40d2-b063-b95eebe773f1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:53.908192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e286c0c-e494-40d2-b063-b95eebe773f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:53.926071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5e286c0c-e494-40d2-b063-b95eebe773f1 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:43:53.927284 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5e286c0c-e494-40d2-b063-b95eebe773f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:53.988448 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1227/2482] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:43:53.997307 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfe7f9-957b-4a96-afbf-1ec08c808d0f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:53.997598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfe7f9-957b-4a96-afbf-1ec08c808d0f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:53.997950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfe7f9-957b-4a96-afbf-1ec08c808d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:54.016607 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-08bfe7f9-957b-4a96-afbf-1ec08c808d0f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:43:54.017837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-08bfe7f9-957b-4a96-afbf-1ec08c808d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:54.025761 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1256/2483] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:43:54.030479 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e286c0c-e494-40d2-b063-b95eebe773f1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:54.040274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e286c0c-e494-40d2-b063-b95eebe773f1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:54.044946 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e62a9-ee1a-4c40-9910-07c8e26de66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:54.045625 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e62a9-ee1a-4c40-9910-07c8e26de66a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:54.045906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e62a9-ee1a-4c40-9910-07c8e26de66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:54.046235 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e62a9-ee1a-4c40-9910-07c8e26de66a admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:54.047134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e62a9-ee1a-4c40-9910-07c8e26de66a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:43:54.047447 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e62a9-ee1a-4c40-9910-07c8e26de66a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:54.049088 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e62a9-ee1a-4c40-9910-07c8e26de66a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:54.061741 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1228/2484] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:43:54.065996 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bfe7f9-957b-4a96-afbf-1ec08c808d0f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:54.075915 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bfe7f9-957b-4a96-afbf-1ec08c808d0f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:54.080721 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e124f5-54e1-43ad-b767-be3afe89c88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:54.081422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e124f5-54e1-43ad-b767-be3afe89c88c admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:54.081708 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e124f5-54e1-43ad-b767-be3afe89c88c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:54.081982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e124f5-54e1-43ad-b767-be3afe89c88c admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:54.082683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e124f5-54e1-43ad-b767-be3afe89c88c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:54.084667 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e124f5-54e1-43ad-b767-be3afe89c88c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:54.195240 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1257/2485] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:43:54 2022] POST /identity/v3/projects => generated 367 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:43:54.196622 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e62a9-ee1a-4c40-9910-07c8e26de66a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:54.207667 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e62a9-ee1a-4c40-9910-07c8e26de66a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:54.249820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:43:54.266402 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b14aa-02cf-42b1-85b4-43cb547b6aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:54.267079 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b14aa-02cf-42b1-85b4-43cb547b6aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:54.267356 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b14aa-02cf-42b1-85b4-43cb547b6aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:54.267685 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b14aa-02cf-42b1-85b4-43cb547b6aa9 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:54.268317 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b14aa-02cf-42b1-85b4-43cb547b6aa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:54.270326 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b14aa-02cf-42b1-85b4-43cb547b6aa9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:54.358278 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1229/2486] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:43:54 2022] POST /identity/v3/users => generated 284 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:43:54.362431 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e124f5-54e1-43ad-b767-be3afe89c88c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:54.373945 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e124f5-54e1-43ad-b767-be3afe89c88c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:54.379805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9eb5b-2e18-45e0-9bd4-ce55fffb508f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:54.380471 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9eb5b-2e18-45e0-9bd4-ce55fffb508f admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:54.380752 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9eb5b-2e18-45e0-9bd4-ce55fffb508f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:54.381136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9eb5b-2e18-45e0-9bd4-ce55fffb508f admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:54.381772 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc9eb5b-2e18-45e0-9bd4-ce55fffb508f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:54.383525 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc9eb5b-2e18-45e0-9bd4-ce55fffb508f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:54.395105 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1258/2487] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:43:54.399175 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b14aa-02cf-42b1-85b4-43cb547b6aa9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:54.416476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b14aa-02cf-42b1-85b4-43cb547b6aa9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:54.420231 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0b0b8-5fdc-4925-907b-53ec93b6d31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:54.420908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0b0b8-5fdc-4925-907b-53ec93b6d31f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:54.421429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0b0b8-5fdc-4925-907b-53ec93b6d31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:54.421865 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0b0b8-5fdc-4925-907b-53ec93b6d31f admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:54.422776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0b0b8-5fdc-4925-907b-53ec93b6d31f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:54.424623 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0b0b8-5fdc-4925-907b-53ec93b6d31f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:54.442303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1230/2488] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:43:54.446333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9eb5b-2e18-45e0-9bd4-ce55fffb508f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:54.451981 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:43:54.457720 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9eb5b-2e18-45e0-9bd4-ce55fffb508f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:54.462662 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfbb55-6439-4d5b-b449-ef42f3572a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:54.463555 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfbb55-6439-4d5b-b449-ef42f3572a30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:54.463836 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfbb55-6439-4d5b-b449-ef42f3572a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:54.465019 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfbb55-6439-4d5b-b449-ef42f3572a30 admin admin] PATH_INFO: `/v3/projects/bfc85651f1d947e3b52f02b4cf8e26b3/users/346ec431db3947ea97f1f2e3b27db0a0/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:54.481811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedfbb55-6439-4d5b-b449-ef42f3572a30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfc85651f1d947e3b52f02b4cf8e26b3, user_id=346ec431db3947ea97f1f2e3b27db0a0, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:54.485137 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedfbb55-6439-4d5b-b449-ef42f3572a30 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:54.586316 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1259/2489] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:43:54 2022] PUT /identity/v3/projects/bfc85651f1d947e3b52f02b4cf8e26b3/users/346ec431db3947ea97f1f2e3b27db0a0/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:43:54.592321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb7d1e-f340-4a2a-8478-1cab00d133d9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:54.592682 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb7d1e-f340-4a2a-8478-1cab00d133d9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:54.593075 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb7d1e-f340-4a2a-8478-1cab00d133d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:54.611390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-07cb7d1e-f340-4a2a-8478-1cab00d133d9 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:43:54.617542 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-07cb7d1e-f340-4a2a-8478-1cab00d133d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:54.673912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1231/2490] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:43:54.683738 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfbb55-6439-4d5b-b449-ef42f3572a30 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:43:54.685459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-dedfbb55-6439-4d5b-b449-ef42f3572a30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:43:54.742159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfbb55-6439-4d5b-b449-ef42f3572a30 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:43:54.745929 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7975ad-b892-4a01-a501-04a99b71c44b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:43:54.746571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7975ad-b892-4a01-a501-04a99b71c44b service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:43:54.746851 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7975ad-b892-4a01-a501-04a99b71c44b service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:43:54.747123 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7975ad-b892-4a01-a501-04a99b71c44b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:43:54.757695 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7975ad-b892-4a01-a501-04a99b71c44b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:43:54.759216 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7975ad-b892-4a01-a501-04a99b71c44b service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:43:54.770360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1260/2491] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:43:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:44:00.618411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c25007-833f-42ec-85e3-7f058e29014d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:00.620494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c25007-833f-42ec-85e3-7f058e29014d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:00.620494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c25007-833f-42ec-85e3-7f058e29014d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:00.648474 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-12c25007-833f-42ec-85e3-7f058e29014d None None] MFA Rules not processed for user `346ec431db3947ea97f1f2e3b27db0a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:00.649738 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-12c25007-833f-42ec-85e3-7f058e29014d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:00.734312 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1232/2492] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:44:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:00.739552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7975ad-b892-4a01-a501-04a99b71c44b service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:00.750959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7975ad-b892-4a01-a501-04a99b71c44b service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:00.756615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96dcc4-1cf9-44b5-a1f8-22b80159b6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:00.757508 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96dcc4-1cf9-44b5-a1f8-22b80159b6a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:00.757855 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96dcc4-1cf9-44b5-a1f8-22b80159b6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:00.758139 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96dcc4-1cf9-44b5-a1f8-22b80159b6a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:00.758830 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96dcc4-1cf9-44b5-a1f8-22b80159b6a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:00.760816 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96dcc4-1cf9-44b5-a1f8-22b80159b6a6 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:00.849835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1261/2493] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:44:00 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:44:00.853880 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c25007-833f-42ec-85e3-7f058e29014d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:00.866260 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c25007-833f-42ec-85e3-7f058e29014d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:00.916334 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:00.927729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c069a8-8d1e-4acd-83ea-6a325ad1881f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:00.928405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c069a8-8d1e-4acd-83ea-6a325ad1881f admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:00.928685 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c069a8-8d1e-4acd-83ea-6a325ad1881f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:00.928958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c069a8-8d1e-4acd-83ea-6a325ad1881f admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:00.929680 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c069a8-8d1e-4acd-83ea-6a325ad1881f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:00.931679 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c069a8-8d1e-4acd-83ea-6a325ad1881f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:01.025557 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1233/2494] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:44:00 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:44:01.030374 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96dcc4-1cf9-44b5-a1f8-22b80159b6a6 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:01.040696 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96dcc4-1cf9-44b5-a1f8-22b80159b6a6 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:01.046189 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7eb34f-71bc-4fbf-ae7a-5349882a6c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:01.046981 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7eb34f-71bc-4fbf-ae7a-5349882a6c14 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:01.047322 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7eb34f-71bc-4fbf-ae7a-5349882a6c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:01.047665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7eb34f-71bc-4fbf-ae7a-5349882a6c14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:01.048492 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7eb34f-71bc-4fbf-ae7a-5349882a6c14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:01.050102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7eb34f-71bc-4fbf-ae7a-5349882a6c14 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:01.062415 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1262/2495] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:44:01.066597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c069a8-8d1e-4acd-83ea-6a325ad1881f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:01.077957 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c069a8-8d1e-4acd-83ea-6a325ad1881f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:01.083087 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298605d5-0397-49c1-906f-9526ebcf71f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:01.083977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298605d5-0397-49c1-906f-9526ebcf71f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:01.084255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298605d5-0397-49c1-906f-9526ebcf71f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:01.084523 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298605d5-0397-49c1-906f-9526ebcf71f2 admin admin] PATH_INFO: `/v3/projects/fd6e957a48f940f4a64e4e525ffb9bff/users/c71fa913b3944742b4a9f89272668294/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:01.100598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298605d5-0397-49c1-906f-9526ebcf71f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd6e957a48f940f4a64e4e525ffb9bff, user_id=c71fa913b3944742b4a9f89272668294, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:01.103198 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298605d5-0397-49c1-906f-9526ebcf71f2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:01.151577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1234/2496] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:44:01 2022] PUT /identity/v3/projects/fd6e957a48f940f4a64e4e525ffb9bff/users/c71fa913b3944742b4a9f89272668294/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:44:01.156708 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7eb34f-71bc-4fbf-ae7a-5349882a6c14 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:01.167584 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7eb34f-71bc-4fbf-ae7a-5349882a6c14 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:01.237411 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab412198-ede3-4696-828b-1b6fed899bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:01.238185 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab412198-ede3-4696-828b-1b6fed899bca admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:01.238532 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab412198-ede3-4696-828b-1b6fed899bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:01.238876 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab412198-ede3-4696-828b-1b6fed899bca admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:01.239710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab412198-ede3-4696-828b-1b6fed899bca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:01.241417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab412198-ede3-4696-828b-1b6fed899bca admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:01.255457 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1263/2497] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:44:01.256716 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298605d5-0397-49c1-906f-9526ebcf71f2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:01.267612 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298605d5-0397-49c1-906f-9526ebcf71f2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:01.273053 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed6a2f-34b0-475e-9ce4-d10b32afdf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:01.273952 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed6a2f-34b0-475e-9ce4-d10b32afdf87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:01.274231 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed6a2f-34b0-475e-9ce4-d10b32afdf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:01.274504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed6a2f-34b0-475e-9ce4-d10b32afdf87 admin admin] PATH_INFO: `/v3/projects/fd6e957a48f940f4a64e4e525ffb9bff/users/c71fa913b3944742b4a9f89272668294/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:01.276547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed6a2f-34b0-475e-9ce4-d10b32afdf87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd6e957a48f940f4a64e4e525ffb9bff, user_id=c71fa913b3944742b4a9f89272668294, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:01.280309 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed6a2f-34b0-475e-9ce4-d10b32afdf87 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:01.295054 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1235/2498] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:44:01 2022] PUT /identity/v3/projects/fd6e957a48f940f4a64e4e525ffb9bff/users/c71fa913b3944742b4a9f89272668294/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:44:07.210551 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb2e31-b8e9-426e-a3cf-43ac00b2aae7 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:07.211081 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb2e31-b8e9-426e-a3cf-43ac00b2aae7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:07.211184 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb2e31-b8e9-426e-a3cf-43ac00b2aae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:07.239031 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9dcb2e31-b8e9-426e-a3cf-43ac00b2aae7 None None] MFA Rules not processed for user `c71fa913b3944742b4a9f89272668294`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:07.240400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9dcb2e31-b8e9-426e-a3cf-43ac00b2aae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:07.273015 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:07.310194 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1264/2499] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:44:07 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:07.323772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed6a2f-34b0-475e-9ce4-d10b32afdf87 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:07.325507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-49ed6a2f-34b0-475e-9ce4-d10b32afdf87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:07.408444 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed6a2f-34b0-475e-9ce4-d10b32afdf87 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:07.414643 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7cf9b-9575-4de5-be88-3824826c0ef8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:07.415295 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7cf9b-9575-4de5-be88-3824826c0ef8 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:07.415574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7cf9b-9575-4de5-be88-3824826c0ef8 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:07.415869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7cf9b-9575-4de5-be88-3824826c0ef8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:07.474087 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7cf9b-9575-4de5-be88-3824826c0ef8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:07.475803 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7cf9b-9575-4de5-be88-3824826c0ef8 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:07.486866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1236/2500] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:44:07 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:44:10.086423 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb708b7c-c73f-4e0a-8abb-7ae71cffc493 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:10.086905 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb708b7c-c73f-4e0a-8abb-7ae71cffc493 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:10.087198 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb708b7c-c73f-4e0a-8abb-7ae71cffc493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:10.116610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-bb708b7c-c73f-4e0a-8abb-7ae71cffc493 None None] MFA Rules not processed for user `346ec431db3947ea97f1f2e3b27db0a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:10.118126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-bb708b7c-c73f-4e0a-8abb-7ae71cffc493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:10.126238 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1265/2501] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:44:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:16.824533 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7cf9b-9575-4de5-be88-3824826c0ef8 service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:16.825951 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-22e7cf9b-9575-4de5-be88-3824826c0ef8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:16.874647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:16.896046 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7cf9b-9575-4de5-be88-3824826c0ef8 service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:16.900857 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195edbf1-af10-4a31-99f1-8bc48bf1885f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:16.901809 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195edbf1-af10-4a31-99f1-8bc48bf1885f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:16.902078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195edbf1-af10-4a31-99f1-8bc48bf1885f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:16.902363 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195edbf1-af10-4a31-99f1-8bc48bf1885f admin admin] PATH_INFO: `/v3/users/c856e25bd5a14c618b90889c9f4f65b4` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:16.903672 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195edbf1-af10-4a31-99f1-8bc48bf1885f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c856e25bd5a14c618b90889c9f4f65b4)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:16.906016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195edbf1-af10-4a31-99f1-8bc48bf1885f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:17.096630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-195edbf1-af10-4a31-99f1-8bc48bf1885f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c856e25bd5a14c618b90889c9f4f65b4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:17.097236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-195edbf1-af10-4a31-99f1-8bc48bf1885f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c856e25bd5a14c618b90889c9f4f65b4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:17.103924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-195edbf1-af10-4a31-99f1-8bc48bf1885f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c856e25bd5a14c618b90889c9f4f65b4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:17.104532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-195edbf1-af10-4a31-99f1-8bc48bf1885f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c856e25bd5a14c618b90889c9f4f65b4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:17.116855 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-195edbf1-af10-4a31-99f1-8bc48bf1885f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c856e25bd5a14c618b90889c9f4f65b4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:17.207958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1237/2502] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:44:16 2022] DELETE /identity/v3/users/c856e25bd5a14c618b90889c9f4f65b4 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:44:17.217803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332f156-f6b4-49ec-a142-2d1c697965e2 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:17.217803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332f156-f6b4-49ec-a142-2d1c697965e2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:17.217803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332f156-f6b4-49ec-a142-2d1c697965e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:17.235131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-f332f156-f6b4-49ec-a142-2d1c697965e2 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:17.236624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f332f156-f6b4-49ec-a142-2d1c697965e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:17.311717 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1266/2503] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:17.322216 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195edbf1-af10-4a31-99f1-8bc48bf1885f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:17.323715 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-195edbf1-af10-4a31-99f1-8bc48bf1885f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:17.390707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195edbf1-af10-4a31-99f1-8bc48bf1885f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:17.394388 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d833e0-f549-4f86-bfc8-1ed917444554 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:17.400711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d833e0-f549-4f86-bfc8-1ed917444554 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:17.400711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d833e0-f549-4f86-bfc8-1ed917444554 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:17.400711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d833e0-f549-4f86-bfc8-1ed917444554 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:17.405629 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d833e0-f549-4f86-bfc8-1ed917444554 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:17.407066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d833e0-f549-4f86-bfc8-1ed917444554 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:17.416734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1238/2504] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:44:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:44:17.878865 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332f156-f6b4-49ec-a142-2d1c697965e2 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:17.880378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f332f156-f6b4-49ec-a142-2d1c697965e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:17.893268 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332f156-f6b4-49ec-a142-2d1c697965e2 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:17.898160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb97c1f-6a63-4ec6-80aa-de5ef5e6b8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:17.899033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb97c1f-6a63-4ec6-80aa-de5ef5e6b8b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:17.899325 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb97c1f-6a63-4ec6-80aa-de5ef5e6b8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:17.899601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb97c1f-6a63-4ec6-80aa-de5ef5e6b8b9 admin admin] PATH_INFO: `/v3/projects/e28f24d8d35f4336be9d43c81bef8b41` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:17.900887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb97c1f-6a63-4ec6-80aa-de5ef5e6b8b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e28f24d8d35f4336be9d43c81bef8b41)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:17.916307 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb97c1f-6a63-4ec6-80aa-de5ef5e6b8b9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:18.077625 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4bb97c1f-6a63-4ec6-80aa-de5ef5e6b8b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e28f24d8d35f4336be9d43c81bef8b41', 'request_id': 'req-4bb97c1f-6a63-4ec6-80aa-de5ef5e6b8b9'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:18.085726 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4bb97c1f-6a63-4ec6-80aa-de5ef5e6b8b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e28f24d8d35f4336be9d43c81bef8b41', 'request_id': 'req-4bb97c1f-6a63-4ec6-80aa-de5ef5e6b8b9'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:18.167544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4bb97c1f-6a63-4ec6-80aa-de5ef5e6b8b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e28f24d8d35f4336be9d43c81bef8b41', 'request_id': 'req-4bb97c1f-6a63-4ec6-80aa-de5ef5e6b8b9'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:18.205901 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4bb97c1f-6a63-4ec6-80aa-de5ef5e6b8b9 admin admin] The token cache is being invalidate because project e28f24d8d35f4336be9d43c81bef8b41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:44:18.205901 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4bb97c1f-6a63-4ec6-80aa-de5ef5e6b8b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:18.205901 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4bb97c1f-6a63-4ec6-80aa-de5ef5e6b8b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:18.205901 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1267/2505] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:44:17 2022] DELETE /identity/v3/projects/e28f24d8d35f4336be9d43c81bef8b41 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:44:18.212189 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d833e0-f549-4f86-bfc8-1ed917444554 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:18.223701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d833e0-f549-4f86-bfc8-1ed917444554 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:18.286815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b51d5-0172-47be-8958-2ee324d77d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:18.287804 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b51d5-0172-47be-8958-2ee324d77d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:18.288252 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b51d5-0172-47be-8958-2ee324d77d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:18.288533 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b51d5-0172-47be-8958-2ee324d77d00 admin admin] PATH_INFO: `/v3/users/d446a323e1434e32b64bccc28be4b137` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:18.289988 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b51d5-0172-47be-8958-2ee324d77d00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d446a323e1434e32b64bccc28be4b137)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:18.292015 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b51d5-0172-47be-8958-2ee324d77d00 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:18.294817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:18.486469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c440e-6834-4e50-85de-a8e08bdde6f0 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:18.487338 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c440e-6834-4e50-85de-a8e08bdde6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:18.487338 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c440e-6834-4e50-85de-a8e08bdde6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:18.490102 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b80b51d5-0172-47be-8958-2ee324d77d00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd446a323e1434e32b64bccc28be4b137'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:18.490944 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b80b51d5-0172-47be-8958-2ee324d77d00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd446a323e1434e32b64bccc28be4b137'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:18.500065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b80b51d5-0172-47be-8958-2ee324d77d00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd446a323e1434e32b64bccc28be4b137'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:18.500934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b80b51d5-0172-47be-8958-2ee324d77d00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd446a323e1434e32b64bccc28be4b137'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:18.506957 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9d4c440e-6834-4e50-85de-a8e08bdde6f0 None None] MFA Rules not processed for user `bbf0ce396885472bb5d225aa009043f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:18.508169 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9d4c440e-6834-4e50-85de-a8e08bdde6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:18.515111 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b80b51d5-0172-47be-8958-2ee324d77d00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd446a323e1434e32b64bccc28be4b137'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:18.554344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:18.571590 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1239/2506] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:44:18 2022] DELETE /identity/v3/users/d446a323e1434e32b64bccc28be4b137 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:44:18.573124 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1268/2507] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jun 7 08:44:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:44:18.589778 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b51d5-0172-47be-8958-2ee324d77d00 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:18.591900 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b80b51d5-0172-47be-8958-2ee324d77d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:18.680982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b51d5-0172-47be-8958-2ee324d77d00 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:18.686631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac697892-676d-4bc1-8fad-c6d925f956a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334dc47ae2e643ccb44d25c6b5906f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:18.687293 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac697892-676d-4bc1-8fad-c6d925f956a1 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:18.687580 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac697892-676d-4bc1-8fad-c6d925f956a1 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:18.687858 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac697892-676d-4bc1-8fad-c6d925f956a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:18.748039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac697892-676d-4bc1-8fad-c6d925f956a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:18.749591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac697892-676d-4bc1-8fad-c6d925f956a1 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:18.757843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1240/2508] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jun 7 08:44:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:44:19.020358 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c440e-6834-4e50-85de-a8e08bdde6f0 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:19.023051 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9d4c440e-6834-4e50-85de-a8e08bdde6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:19.044116 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c440e-6834-4e50-85de-a8e08bdde6f0 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:19.128575 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39fddd-cdcb-4122-b9a7-9dd6444abc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:19.129488 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39fddd-cdcb-4122-b9a7-9dd6444abc4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:19.129785 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39fddd-cdcb-4122-b9a7-9dd6444abc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:19.130043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39fddd-cdcb-4122-b9a7-9dd6444abc4a admin admin] PATH_INFO: `/v3/projects/de670bb8290d488d9abd42634361435c` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:19.131285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39fddd-cdcb-4122-b9a7-9dd6444abc4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=de670bb8290d488d9abd42634361435c)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:19.133253 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39fddd-cdcb-4122-b9a7-9dd6444abc4a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:19.284922 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ab39fddd-cdcb-4122-b9a7-9dd6444abc4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de670bb8290d488d9abd42634361435c', 'request_id': 'req-ab39fddd-cdcb-4122-b9a7-9dd6444abc4a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:19.292425 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ab39fddd-cdcb-4122-b9a7-9dd6444abc4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de670bb8290d488d9abd42634361435c', 'request_id': 'req-ab39fddd-cdcb-4122-b9a7-9dd6444abc4a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:19.379421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ab39fddd-cdcb-4122-b9a7-9dd6444abc4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de670bb8290d488d9abd42634361435c', 'request_id': 'req-ab39fddd-cdcb-4122-b9a7-9dd6444abc4a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:19.393938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ab39fddd-cdcb-4122-b9a7-9dd6444abc4a admin admin] The token cache is being invalidate because project de670bb8290d488d9abd42634361435c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:44:19.395478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ab39fddd-cdcb-4122-b9a7-9dd6444abc4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:19.397453 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ab39fddd-cdcb-4122-b9a7-9dd6444abc4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:19.402416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1269/2509] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:44:19 2022] DELETE /identity/v3/projects/de670bb8290d488d9abd42634361435c => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:44:19.404160 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8e003-4d73-4d2e-a08c-b871a48d3fb0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:19.404612 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8e003-4d73-4d2e-a08c-b871a48d3fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:19.405100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8e003-4d73-4d2e-a08c-b871a48d3fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:19.440882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-27c8e003-4d73-4d2e-a08c-b871a48d3fb0 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:19.442719 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-27c8e003-4d73-4d2e-a08c-b871a48d3fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:19.482647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:19.565217 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1241/2510] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:19.570355 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d850a6-5316-4ae9-a2a9-aa6b825dd849 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:19.570937 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d850a6-5316-4ae9-a2a9-aa6b825dd849 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:19.570937 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d850a6-5316-4ae9-a2a9-aa6b825dd849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:19.599950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c2d850a6-5316-4ae9-a2a9-aa6b825dd849 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:19.602095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c2d850a6-5316-4ae9-a2a9-aa6b825dd849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:19.623548 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1270/2511] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:19.624837 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8e003-4d73-4d2e-a08c-b871a48d3fb0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:19.637110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8e003-4d73-4d2e-a08c-b871a48d3fb0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:19.649522 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d850a6-5316-4ae9-a2a9-aa6b825dd849 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:19.651578 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c2d850a6-5316-4ae9-a2a9-aa6b825dd849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:19.656863 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924213b2-534d-4eea-8962-99f95adb8258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:19.658218 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924213b2-534d-4eea-8962-99f95adb8258 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:19.658561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924213b2-534d-4eea-8962-99f95adb8258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:19.658834 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924213b2-534d-4eea-8962-99f95adb8258 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:19.659706 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924213b2-534d-4eea-8962-99f95adb8258 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:44:19.660098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924213b2-534d-4eea-8962-99f95adb8258 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:19.662000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924213b2-534d-4eea-8962-99f95adb8258 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:19.679557 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1242/2512] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:44:19.686336 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924213b2-534d-4eea-8962-99f95adb8258 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:19.701074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924213b2-534d-4eea-8962-99f95adb8258 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:19.710377 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34acac19-64f9-425d-af98-87bd3a26d064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:19.711272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acac19-64f9-425d-af98-87bd3a26d064 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:19.711793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acac19-64f9-425d-af98-87bd3a26d064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:19.712501 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acac19-64f9-425d-af98-87bd3a26d064 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:19.713443 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34acac19-64f9-425d-af98-87bd3a26d064 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:19.717825 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34acac19-64f9-425d-af98-87bd3a26d064 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:19.813179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d850a6-5316-4ae9-a2a9-aa6b825dd849 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:19.820533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030d568-ccca-453c-a2ed-9a1f12fc7169 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:19.821326 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030d568-ccca-453c-a2ed-9a1f12fc7169 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:19.821681 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030d568-ccca-453c-a2ed-9a1f12fc7169 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:19.822032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030d568-ccca-453c-a2ed-9a1f12fc7169 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:19.824758 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7030d568-ccca-453c-a2ed-9a1f12fc7169 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:19.858385 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030d568-ccca-453c-a2ed-9a1f12fc7169 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:19.860973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030d568-ccca-453c-a2ed-9a1f12fc7169 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:19.869846 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1243/2513] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:44:19 2022] POST /identity/v3/projects => generated 401 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:44:19.871515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34acac19-64f9-425d-af98-87bd3a26d064 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:19.878438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:19.893221 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1271/2514] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:44:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:44:19.977554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34acac19-64f9-425d-af98-87bd3a26d064 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:19.988825 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f2be5-5aba-474e-8f1a-50750aedba4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:19.989749 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f2be5-5aba-474e-8f1a-50750aedba4f admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:19.990133 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f2be5-5aba-474e-8f1a-50750aedba4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:19.990522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f2be5-5aba-474e-8f1a-50750aedba4f admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:19.991343 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f2be5-5aba-474e-8f1a-50750aedba4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:19.993452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f2be5-5aba-474e-8f1a-50750aedba4f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:20.155179 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1244/2515] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:44:19 2022] POST /identity/v3/users => generated 301 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:44:20.155715 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030d568-ccca-453c-a2ed-9a1f12fc7169 service nova] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:20.167180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030d568-ccca-453c-a2ed-9a1f12fc7169 service nova] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:20.190474 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f5a85-7f7f-4972-97f1-9496bef9cde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:20.191425 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f5a85-7f7f-4972-97f1-9496bef9cde0 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:20.191846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f5a85-7f7f-4972-97f1-9496bef9cde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:20.192348 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f5a85-7f7f-4972-97f1-9496bef9cde0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:20.195518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f5a85-7f7f-4972-97f1-9496bef9cde0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:20.197524 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f5a85-7f7f-4972-97f1-9496bef9cde0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:20.216541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f2be5-5aba-474e-8f1a-50750aedba4f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:20.223805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1272/2516] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:44:20.228226 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f2be5-5aba-474e-8f1a-50750aedba4f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:20.234821 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d614a-ba91-4acb-95b6-8f57c4316cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:20.237327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d614a-ba91-4acb-95b6-8f57c4316cac admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:20.237327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d614a-ba91-4acb-95b6-8f57c4316cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:20.237327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d614a-ba91-4acb-95b6-8f57c4316cac admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:20.237327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d614a-ba91-4acb-95b6-8f57c4316cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:20.238536 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d614a-ba91-4acb-95b6-8f57c4316cac admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:20.251744 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1245/2517] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:44:20.256102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f5a85-7f7f-4972-97f1-9496bef9cde0 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:20.273970 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f5a85-7f7f-4972-97f1-9496bef9cde0 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:20.280486 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7939ac9-2cd4-4455-87ab-b46c416ca359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:20.282987 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7939ac9-2cd4-4455-87ab-b46c416ca359 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:20.282987 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7939ac9-2cd4-4455-87ab-b46c416ca359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:20.282987 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7939ac9-2cd4-4455-87ab-b46c416ca359 admin admin] PATH_INFO: `/v3/projects/89fcc5ff6d9e4bac941fa33763df3b5a/users/53ed1374973341cdb5b719cd689d7b1a/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:20.303880 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7939ac9-2cd4-4455-87ab-b46c416ca359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89fcc5ff6d9e4bac941fa33763df3b5a, user_id=53ed1374973341cdb5b719cd689d7b1a, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:20.307359 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7939ac9-2cd4-4455-87ab-b46c416ca359 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:20.387556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1273/2518] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:44:20 2022] PUT /identity/v3/projects/89fcc5ff6d9e4bac941fa33763df3b5a/users/53ed1374973341cdb5b719cd689d7b1a/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:44:20.398734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47598cca-30d5-4a77-9979-d906b68d22b1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:20.399172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47598cca-30d5-4a77-9979-d906b68d22b1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:20.399702 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47598cca-30d5-4a77-9979-d906b68d22b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:20.471064 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-47598cca-30d5-4a77-9979-d906b68d22b1 None None] MFA Rules not processed for user `53ed1374973341cdb5b719cd689d7b1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:20.482146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-47598cca-30d5-4a77-9979-d906b68d22b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:20.588899 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:20.637681 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1246/2519] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:44:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:20.638464 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7939ac9-2cd4-4455-87ab-b46c416ca359 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:20.741289 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7939ac9-2cd4-4455-87ab-b46c416ca359 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:20.746956 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00559b54-6d38-4b85-8c0f-faf1c9ad9c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:20.747723 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00559b54-6d38-4b85-8c0f-faf1c9ad9c91 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:20.748003 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00559b54-6d38-4b85-8c0f-faf1c9ad9c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:20.748302 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00559b54-6d38-4b85-8c0f-faf1c9ad9c91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:20.749278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00559b54-6d38-4b85-8c0f-faf1c9ad9c91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:20.751159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00559b54-6d38-4b85-8c0f-faf1c9ad9c91 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:20.845130 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1274/2520] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:44:20 2022] POST /identity/v3/projects => generated 401 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:44:20.845387 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47598cca-30d5-4a77-9979-d906b68d22b1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:20.859309 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47598cca-30d5-4a77-9979-d906b68d22b1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:20.944111 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb576791-efe8-4411-abef-4fd9a81ebe50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:20.945015 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb576791-efe8-4411-abef-4fd9a81ebe50 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:20.945391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb576791-efe8-4411-abef-4fd9a81ebe50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:20.945677 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb576791-efe8-4411-abef-4fd9a81ebe50 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:20.946394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb576791-efe8-4411-abef-4fd9a81ebe50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:20.948346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb576791-efe8-4411-abef-4fd9a81ebe50 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:21.063151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1247/2521] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:44:20 2022] POST /identity/v3/users => generated 307 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:44:21.064387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00559b54-6d38-4b85-8c0f-faf1c9ad9c91 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:21.076156 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00559b54-6d38-4b85-8c0f-faf1c9ad9c91 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:21.085138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a4219-7107-4e3e-a349-5e141ea0188d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:21.085850 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a4219-7107-4e3e-a349-5e141ea0188d admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:21.086417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a4219-7107-4e3e-a349-5e141ea0188d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:21.086417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a4219-7107-4e3e-a349-5e141ea0188d admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:21.087213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a4219-7107-4e3e-a349-5e141ea0188d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:21.089482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a4219-7107-4e3e-a349-5e141ea0188d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:21.104423 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1275/2522] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:44:21.108002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb576791-efe8-4411-abef-4fd9a81ebe50 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:21.118509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb576791-efe8-4411-abef-4fd9a81ebe50 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:21.124157 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abfe9f-9bf7-4d61-9840-da66bc6314e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:21.125237 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abfe9f-9bf7-4d61-9840-da66bc6314e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:21.127823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abfe9f-9bf7-4d61-9840-da66bc6314e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:21.127823 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abfe9f-9bf7-4d61-9840-da66bc6314e1 admin admin] PATH_INFO: `/v3/projects/3def6a7dd3444431a8f0bee157ee7439/users/3f3fe1971b3c4ac09802e5ce535937b1/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:21.147660 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0abfe9f-9bf7-4d61-9840-da66bc6314e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3def6a7dd3444431a8f0bee157ee7439, user_id=3f3fe1971b3c4ac09802e5ce535937b1, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:21.151271 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0abfe9f-9bf7-4d61-9840-da66bc6314e1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:21.250668 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1248/2523] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:44:21 2022] PUT /identity/v3/projects/3def6a7dd3444431a8f0bee157ee7439/users/3f3fe1971b3c4ac09802e5ce535937b1/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:44:21.254598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a4219-7107-4e3e-a349-5e141ea0188d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:21.266818 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a4219-7107-4e3e-a349-5e141ea0188d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:21.349689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3059e-9018-4a89-b368-083bc32ac88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:21.350371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3059e-9018-4a89-b368-083bc32ac88e admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:21.350652 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3059e-9018-4a89-b368-083bc32ac88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:21.350927 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3059e-9018-4a89-b368-083bc32ac88e admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:21.351663 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff3059e-9018-4a89-b368-083bc32ac88e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:21.353559 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff3059e-9018-4a89-b368-083bc32ac88e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:21.366323 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1276/2524] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:44:21.370250 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abfe9f-9bf7-4d61-9840-da66bc6314e1 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:21.383908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abfe9f-9bf7-4d61-9840-da66bc6314e1 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:21.392216 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc08d9-291a-4fab-99e8-de0f4eab2d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:21.393263 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc08d9-291a-4fab-99e8-de0f4eab2d12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:21.393612 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc08d9-291a-4fab-99e8-de0f4eab2d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:21.393963 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc08d9-291a-4fab-99e8-de0f4eab2d12 admin admin] PATH_INFO: `/v3/projects/3def6a7dd3444431a8f0bee157ee7439/users/3f3fe1971b3c4ac09802e5ce535937b1/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:21.396053 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc08d9-291a-4fab-99e8-de0f4eab2d12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3def6a7dd3444431a8f0bee157ee7439, user_id=3f3fe1971b3c4ac09802e5ce535937b1, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:21.399050 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc08d9-291a-4fab-99e8-de0f4eab2d12 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:21.415205 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1249/2525] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:44:21 2022] PUT /identity/v3/projects/3def6a7dd3444431a8f0bee157ee7439/users/3f3fe1971b3c4ac09802e5ce535937b1/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:44:21.427883 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf4e5d-1855-4949-a22f-f1c9b6576f01 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:21.427883 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf4e5d-1855-4949-a22f-f1c9b6576f01 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:21.427883 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf4e5d-1855-4949-a22f-f1c9b6576f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:21.459109 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1acf4e5d-1855-4949-a22f-f1c9b6576f01 None None] MFA Rules not processed for user `3f3fe1971b3c4ac09802e5ce535937b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:21.460395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1acf4e5d-1855-4949-a22f-f1c9b6576f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:21.491934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:21.556822 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1277/2526] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:21.571195 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc08d9-291a-4fab-99e8-de0f4eab2d12 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:21.573626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f5bc08d9-291a-4fab-99e8-de0f4eab2d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:21.687785 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc08d9-291a-4fab-99e8-de0f4eab2d12 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:21.698417 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9a240-e1dd-4510-9d11-0ec380c1bdfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:21.699118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9a240-e1dd-4510-9d11-0ec380c1bdfb service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:21.699397 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9a240-e1dd-4510-9d11-0ec380c1bdfb service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:21.699671 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9a240-e1dd-4510-9d11-0ec380c1bdfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:21.716867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9a240-e1dd-4510-9d11-0ec380c1bdfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:21.717864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9a240-e1dd-4510-9d11-0ec380c1bdfb service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:21.736226 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1250/2527] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:44:21 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:44:22.044673 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf4e5d-1855-4949-a22f-f1c9b6576f01 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:22.046166 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1acf4e5d-1855-4949-a22f-f1c9b6576f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:22.062426 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf4e5d-1855-4949-a22f-f1c9b6576f01 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:22.065907 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf4795e-7a9c-4bd1-9ffa-1770eb15d292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:22.066553 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4795e-7a9c-4bd1-9ffa-1770eb15d292 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:22.066835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4795e-7a9c-4bd1-9ffa-1770eb15d292 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:22.067209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4795e-7a9c-4bd1-9ffa-1770eb15d292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:22.132749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf4795e-7a9c-4bd1-9ffa-1770eb15d292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:22.156539 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf4795e-7a9c-4bd1-9ffa-1770eb15d292 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:22.156539 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1278/2528] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:44:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:44:27.197317 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9a240-e1dd-4510-9d11-0ec380c1bdfb service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:27.208818 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9a240-e1dd-4510-9d11-0ec380c1bdfb service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:27.213237 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37135ed6-f5c4-4d78-9b40-27b8edb11f90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:27.214427 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37135ed6-f5c4-4d78-9b40-27b8edb11f90 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:27.214427 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37135ed6-f5c4-4d78-9b40-27b8edb11f90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:27.214658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37135ed6-f5c4-4d78-9b40-27b8edb11f90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:27.216960 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-37135ed6-f5c4-4d78-9b40-27b8edb11f90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:27.269150 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:27.291066 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37135ed6-f5c4-4d78-9b40-27b8edb11f90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:27.293257 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37135ed6-f5c4-4d78-9b40-27b8edb11f90 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:27.312181 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1251/2529] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:44:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:44:27.505402 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf4795e-7a9c-4bd1-9ffa-1770eb15d292 service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:27.592989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf4795e-7a9c-4bd1-9ffa-1770eb15d292 service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:27.600591 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb9ab0-8d77-456a-b289-ca6cfb8b8029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:27.602178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb9ab0-8d77-456a-b289-ca6cfb8b8029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:27.602701 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb9ab0-8d77-456a-b289-ca6cfb8b8029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:27.603203 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb9ab0-8d77-456a-b289-ca6cfb8b8029 admin admin] PATH_INFO: `/v3/users/53ed1374973341cdb5b719cd689d7b1a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:27.605475 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb9ab0-8d77-456a-b289-ca6cfb8b8029 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53ed1374973341cdb5b719cd689d7b1a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:27.610210 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb9ab0-8d77-456a-b289-ca6cfb8b8029 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:27.813974 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e2eb9ab0-8d77-456a-b289-ca6cfb8b8029 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53ed1374973341cdb5b719cd689d7b1a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:27.816004 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e2eb9ab0-8d77-456a-b289-ca6cfb8b8029 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53ed1374973341cdb5b719cd689d7b1a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:27.825229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e2eb9ab0-8d77-456a-b289-ca6cfb8b8029 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53ed1374973341cdb5b719cd689d7b1a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:27.826238 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e2eb9ab0-8d77-456a-b289-ca6cfb8b8029 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53ed1374973341cdb5b719cd689d7b1a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:27.831537 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:27.899446 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e2eb9ab0-8d77-456a-b289-ca6cfb8b8029 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53ed1374973341cdb5b719cd689d7b1a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:27.921905 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1279/2530] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:44:27 2022] DELETE /identity/v3/users/53ed1374973341cdb5b719cd689d7b1a => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:44:27.926280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28928b30-cb12-4b47-84f2-6fe90c7533b4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:27.926550 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28928b30-cb12-4b47-84f2-6fe90c7533b4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:27.926832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28928b30-cb12-4b47-84f2-6fe90c7533b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:27.960327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-28928b30-cb12-4b47-84f2-6fe90c7533b4 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:27.962078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-28928b30-cb12-4b47-84f2-6fe90c7533b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:28.056538 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1252/2531] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:44:28.065514 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb9ab0-8d77-456a-b289-ca6cfb8b8029 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:28.068783 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e2eb9ab0-8d77-456a-b289-ca6cfb8b8029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:28.159175 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb9ab0-8d77-456a-b289-ca6cfb8b8029 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:28.163506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff6207-5776-4fa9-a017-e0def20e2bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:28.164267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff6207-5776-4fa9-a017-e0def20e2bce service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:28.164648 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff6207-5776-4fa9-a017-e0def20e2bce service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:28.165048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff6207-5776-4fa9-a017-e0def20e2bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:28.179333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff6207-5776-4fa9-a017-e0def20e2bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:28.180966 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff6207-5776-4fa9-a017-e0def20e2bce service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:28.195228 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1280/2532] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:44:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:44:28.446348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28928b30-cb12-4b47-84f2-6fe90c7533b4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:28.447884 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-28928b30-cb12-4b47-84f2-6fe90c7533b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:28.464983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28928b30-cb12-4b47-84f2-6fe90c7533b4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:28.471290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba7157-cd57-4e7b-a7c1-4f631730e843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:28.472249 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba7157-cd57-4e7b-a7c1-4f631730e843 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:28.472621 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba7157-cd57-4e7b-a7c1-4f631730e843 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:28.473043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba7157-cd57-4e7b-a7c1-4f631730e843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:28.475183 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-25ba7157-cd57-4e7b-a7c1-4f631730e843 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:28.487206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba7157-cd57-4e7b-a7c1-4f631730e843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:28.489299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba7157-cd57-4e7b-a7c1-4f631730e843 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:28.503822 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1253/2533] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:44:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:44:28.893043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff6207-5776-4fa9-a017-e0def20e2bce service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:28.893043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9cff6207-5776-4fa9-a017-e0def20e2bce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:28.913829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff6207-5776-4fa9-a017-e0def20e2bce service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:28.920829 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a54b26-3faa-4f63-85ed-9b6f3bf18b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:28.921747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a54b26-3faa-4f63-85ed-9b6f3bf18b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:28.922148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a54b26-3faa-4f63-85ed-9b6f3bf18b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:28.922382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a54b26-3faa-4f63-85ed-9b6f3bf18b37 admin admin] PATH_INFO: `/v3/projects/89fcc5ff6d9e4bac941fa33763df3b5a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:28.923846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a54b26-3faa-4f63-85ed-9b6f3bf18b37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89fcc5ff6d9e4bac941fa33763df3b5a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:28.925892 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a54b26-3faa-4f63-85ed-9b6f3bf18b37 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:29.518967 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-05a54b26-3faa-4f63-85ed-9b6f3bf18b37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89fcc5ff6d9e4bac941fa33763df3b5a', 'request_id': 'req-05a54b26-3faa-4f63-85ed-9b6f3bf18b37'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:29.525065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-05a54b26-3faa-4f63-85ed-9b6f3bf18b37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89fcc5ff6d9e4bac941fa33763df3b5a', 'request_id': 'req-05a54b26-3faa-4f63-85ed-9b6f3bf18b37'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:29.620199 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-05a54b26-3faa-4f63-85ed-9b6f3bf18b37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89fcc5ff6d9e4bac941fa33763df3b5a', 'request_id': 'req-05a54b26-3faa-4f63-85ed-9b6f3bf18b37'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:29.634970 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-05a54b26-3faa-4f63-85ed-9b6f3bf18b37 admin admin] The token cache is being invalidate because project 89fcc5ff6d9e4bac941fa33763df3b5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:44:29.635627 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-05a54b26-3faa-4f63-85ed-9b6f3bf18b37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:29.636505 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-05a54b26-3faa-4f63-85ed-9b6f3bf18b37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:29.638928 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1281/2534] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:44:28 2022] DELETE /identity/v3/projects/89fcc5ff6d9e4bac941fa33763df3b5a => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:44:29.643748 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba7157-cd57-4e7b-a7c1-4f631730e843 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:29.666854 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba7157-cd57-4e7b-a7c1-4f631730e843 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:29.753199 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd752c2-35f8-4fd1-880b-583140d9e4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:29.754358 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd752c2-35f8-4fd1-880b-583140d9e4fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:29.754358 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd752c2-35f8-4fd1-880b-583140d9e4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:29.754688 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd752c2-35f8-4fd1-880b-583140d9e4fa admin admin] PATH_INFO: `/v3/users/3f3fe1971b3c4ac09802e5ce535937b1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:29.756373 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd752c2-35f8-4fd1-880b-583140d9e4fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f3fe1971b3c4ac09802e5ce535937b1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:29.758303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd752c2-35f8-4fd1-880b-583140d9e4fa admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:29.947078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7bd752c2-35f8-4fd1-880b-583140d9e4fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f3fe1971b3c4ac09802e5ce535937b1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:29.947748 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7bd752c2-35f8-4fd1-880b-583140d9e4fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f3fe1971b3c4ac09802e5ce535937b1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:29.973698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7bd752c2-35f8-4fd1-880b-583140d9e4fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f3fe1971b3c4ac09802e5ce535937b1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:29.974923 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7bd752c2-35f8-4fd1-880b-583140d9e4fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f3fe1971b3c4ac09802e5ce535937b1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:29.989781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:29.992459 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7bd752c2-35f8-4fd1-880b-583140d9e4fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f3fe1971b3c4ac09802e5ce535937b1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:30.096106 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1254/2535] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:44:29 2022] DELETE /identity/v3/users/3f3fe1971b3c4ac09802e5ce535937b1 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:44:30.586379 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a54b26-3faa-4f63-85ed-9b6f3bf18b37 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:30.587982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-05a54b26-3faa-4f63-85ed-9b6f3bf18b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:30.619745 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a54b26-3faa-4f63-85ed-9b6f3bf18b37 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:30.692129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64324f07-490a-4b30-95a9-21fd7b2a1221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:30.693077 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64324f07-490a-4b30-95a9-21fd7b2a1221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:30.693362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64324f07-490a-4b30-95a9-21fd7b2a1221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:30.693788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64324f07-490a-4b30-95a9-21fd7b2a1221 admin admin] PATH_INFO: `/v3/projects/3def6a7dd3444431a8f0bee157ee7439` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:30.694982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64324f07-490a-4b30-95a9-21fd7b2a1221 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3def6a7dd3444431a8f0bee157ee7439)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:30.697298 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64324f07-490a-4b30-95a9-21fd7b2a1221 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:30.704522 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:30.816384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-64324f07-490a-4b30-95a9-21fd7b2a1221 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3def6a7dd3444431a8f0bee157ee7439', 'request_id': 'req-64324f07-490a-4b30-95a9-21fd7b2a1221'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:30.822862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-64324f07-490a-4b30-95a9-21fd7b2a1221 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3def6a7dd3444431a8f0bee157ee7439', 'request_id': 'req-64324f07-490a-4b30-95a9-21fd7b2a1221'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:30.870541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-64324f07-490a-4b30-95a9-21fd7b2a1221 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3def6a7dd3444431a8f0bee157ee7439', 'request_id': 'req-64324f07-490a-4b30-95a9-21fd7b2a1221'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:30.881249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-64324f07-490a-4b30-95a9-21fd7b2a1221 admin admin] The token cache is being invalidate because project 3def6a7dd3444431a8f0bee157ee7439 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:44:30.881532 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-64324f07-490a-4b30-95a9-21fd7b2a1221 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:30.883536 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-64324f07-490a-4b30-95a9-21fd7b2a1221 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:30.883777 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1282/2536] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:44:30 2022] DELETE /identity/v3/projects/3def6a7dd3444431a8f0bee157ee7439 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:44:30.889223 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75439b-4375-459c-a16d-e99101d61950 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:30.889729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75439b-4375-459c-a16d-e99101d61950 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:30.889871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75439b-4375-459c-a16d-e99101d61950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:30.909132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0d75439b-4375-459c-a16d-e99101d61950 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:30.910340 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0d75439b-4375-459c-a16d-e99101d61950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:30.979136 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1255/2537] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:44:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:30.981699 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1615538-33d3-46cf-859e-88164276b236 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:30.981990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1615538-33d3-46cf-859e-88164276b236 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:30.982308 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1615538-33d3-46cf-859e-88164276b236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:31.002528 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d1615538-33d3-46cf-859e-88164276b236 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:31.012826 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d1615538-33d3-46cf-859e-88164276b236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:31.012826 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1283/2538] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:44:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:31.016544 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75439b-4375-459c-a16d-e99101d61950 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:31.026482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75439b-4375-459c-a16d-e99101d61950 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:31.031890 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ac377-0143-46f4-9ea4-699c1dc0e856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:31.032554 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ac377-0143-46f4-9ea4-699c1dc0e856 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:31.032841 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ac377-0143-46f4-9ea4-699c1dc0e856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:31.033151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ac377-0143-46f4-9ea4-699c1dc0e856 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:31.033872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ac377-0143-46f4-9ea4-699c1dc0e856 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:44:31.034178 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ac377-0143-46f4-9ea4-699c1dc0e856 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:31.035781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ac377-0143-46f4-9ea4-699c1dc0e856 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:31.047326 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1256/2539] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:44:31.051622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1615538-33d3-46cf-859e-88164276b236 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:31.061450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1615538-33d3-46cf-859e-88164276b236 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:31.066213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058c423-a873-4b8e-8f22-5f73f7befe58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:31.066897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058c423-a873-4b8e-8f22-5f73f7befe58 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:31.067176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058c423-a873-4b8e-8f22-5f73f7befe58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:31.067448 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058c423-a873-4b8e-8f22-5f73f7befe58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:31.068139 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058c423-a873-4b8e-8f22-5f73f7befe58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:31.070123 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058c423-a873-4b8e-8f22-5f73f7befe58 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:31.186014 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1284/2540] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:44:31 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:44:31.190851 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ac377-0143-46f4-9ea4-699c1dc0e856 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:31.202836 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ac377-0143-46f4-9ea4-699c1dc0e856 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:31.259669 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b8073-3712-4003-bbef-407c72b6e602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:31.261319 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b8073-3712-4003-bbef-407c72b6e602 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:31.261319 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b8073-3712-4003-bbef-407c72b6e602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:31.262693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b8073-3712-4003-bbef-407c72b6e602 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:31.267933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b8073-3712-4003-bbef-407c72b6e602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:31.267933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b8073-3712-4003-bbef-407c72b6e602 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:31.385968 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1257/2541] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:44:31 2022] POST /identity/v3/users => generated 287 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:44:31.391998 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058c423-a873-4b8e-8f22-5f73f7befe58 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:31.403775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058c423-a873-4b8e-8f22-5f73f7befe58 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:31.409750 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9445f-e430-4223-92e2-0d39e14b4a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:31.410741 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9445f-e430-4223-92e2-0d39e14b4a5a admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:31.410741 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9445f-e430-4223-92e2-0d39e14b4a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:31.414915 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9445f-e430-4223-92e2-0d39e14b4a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:31.414915 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9445f-e430-4223-92e2-0d39e14b4a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:31.414915 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9445f-e430-4223-92e2-0d39e14b4a5a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:31.425653 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1285/2542] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:44:31.429651 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b8073-3712-4003-bbef-407c72b6e602 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:31.443137 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b8073-3712-4003-bbef-407c72b6e602 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:31.445678 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7b078-4fd7-4f54-81c0-d26008ece79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:31.446381 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7b078-4fd7-4f54-81c0-d26008ece79b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:31.446667 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7b078-4fd7-4f54-81c0-d26008ece79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:31.446945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7b078-4fd7-4f54-81c0-d26008ece79b admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:31.447708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7b078-4fd7-4f54-81c0-d26008ece79b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:31.450048 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7b078-4fd7-4f54-81c0-d26008ece79b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:31.461646 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1258/2543] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:44:31.466182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9445f-e430-4223-92e2-0d39e14b4a5a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:31.479463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9445f-e430-4223-92e2-0d39e14b4a5a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:31.485178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5d60e-e10d-41cb-93fb-3b309b4da74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:31.488867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5d60e-e10d-41cb-93fb-3b309b4da74b admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:31.488867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5d60e-e10d-41cb-93fb-3b309b4da74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:31.488867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5d60e-e10d-41cb-93fb-3b309b4da74b admin admin] PATH_INFO: `/v3/projects/e14f44aca77049c993e39c8373aaeaaa/users/a18749b8d6884acdbb7f6b7d816bb05d/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:31.509064 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5d60e-e10d-41cb-93fb-3b309b4da74b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e14f44aca77049c993e39c8373aaeaaa, user_id=a18749b8d6884acdbb7f6b7d816bb05d, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:31.511712 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5d60e-e10d-41cb-93fb-3b309b4da74b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:31.576456 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1286/2544] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:44:31 2022] PUT /identity/v3/projects/e14f44aca77049c993e39c8373aaeaaa/users/a18749b8d6884acdbb7f6b7d816bb05d/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:44:31.581150 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50439e23-ae13-40d8-86f8-abb5d807d74d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:31.581294 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50439e23-ae13-40d8-86f8-abb5d807d74d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:31.581574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50439e23-ae13-40d8-86f8-abb5d807d74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:31.600497 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-50439e23-ae13-40d8-86f8-abb5d807d74d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:31.601737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-50439e23-ae13-40d8-86f8-abb5d807d74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:31.657053 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:31.671689 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1259/2545] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:31.682028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5d60e-e10d-41cb-93fb-3b309b4da74b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:31.683891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-25f5d60e-e10d-41cb-93fb-3b309b4da74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:31.743868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5d60e-e10d-41cb-93fb-3b309b4da74b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:31.749049 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcb794-fc05-493c-9b37-91c68ca86397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:31.749948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcb794-fc05-493c-9b37-91c68ca86397 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:31.750284 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcb794-fc05-493c-9b37-91c68ca86397 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:31.750628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcb794-fc05-493c-9b37-91c68ca86397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:31.760808 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcb794-fc05-493c-9b37-91c68ca86397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:31.762878 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcb794-fc05-493c-9b37-91c68ca86397 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:31.773053 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1287/2546] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:44:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:44:33.134885 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c96198-0a16-49f5-b009-ecef72464bd6 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:33.135353 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c96198-0a16-49f5-b009-ecef72464bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:33.135456 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c96198-0a16-49f5-b009-ecef72464bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:33.164440 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f5c96198-0a16-49f5-b009-ecef72464bd6 None None] MFA Rules not processed for user `a18749b8d6884acdbb7f6b7d816bb05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:33.165817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f5c96198-0a16-49f5-b009-ecef72464bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:33.243900 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1260/2547] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:44:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:33.249027 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcb794-fc05-493c-9b37-91c68ca86397 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:33.258669 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcb794-fc05-493c-9b37-91c68ca86397 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:33.263209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c13df3-a963-4bb7-9076-761154b7eb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:33.263848 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c13df3-a963-4bb7-9076-761154b7eb35 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:33.264126 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c13df3-a963-4bb7-9076-761154b7eb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:33.264399 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c13df3-a963-4bb7-9076-761154b7eb35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:33.265112 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c13df3-a963-4bb7-9076-761154b7eb35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:33.267004 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c13df3-a963-4bb7-9076-761154b7eb35 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:33.380774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1288/2548] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:44:33 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:44:33.384730 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c96198-0a16-49f5-b009-ecef72464bd6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:33.395564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c96198-0a16-49f5-b009-ecef72464bd6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:33.452334 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69346c-bdd0-43fc-b1a4-05977e147324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:33.453013 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69346c-bdd0-43fc-b1a4-05977e147324 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:33.453305 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69346c-bdd0-43fc-b1a4-05977e147324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:33.453579 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69346c-bdd0-43fc-b1a4-05977e147324 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:33.454286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69346c-bdd0-43fc-b1a4-05977e147324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:33.456202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69346c-bdd0-43fc-b1a4-05977e147324 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:33.556469 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1261/2549] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:44:33 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:44:33.560721 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c13df3-a963-4bb7-9076-761154b7eb35 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:33.571089 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c13df3-a963-4bb7-9076-761154b7eb35 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:33.575884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53427e44-d0fc-4668-8e59-d017272c815b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:33.576531 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53427e44-d0fc-4668-8e59-d017272c815b admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:33.576808 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53427e44-d0fc-4668-8e59-d017272c815b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:33.577107 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53427e44-d0fc-4668-8e59-d017272c815b admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:33.577943 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53427e44-d0fc-4668-8e59-d017272c815b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:33.579619 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53427e44-d0fc-4668-8e59-d017272c815b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:33.584908 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:33.591241 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1289/2550] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:44:33.595391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69346c-bdd0-43fc-b1a4-05977e147324 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:33.606207 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69346c-bdd0-43fc-b1a4-05977e147324 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:33.610981 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2159bb-bc70-435c-9ef9-7e1bcfb397b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:33.611903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2159bb-bc70-435c-9ef9-7e1bcfb397b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:33.612181 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2159bb-bc70-435c-9ef9-7e1bcfb397b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:33.612454 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2159bb-bc70-435c-9ef9-7e1bcfb397b4 admin admin] PATH_INFO: `/v3/projects/72a0aec02ff04382b23a50fc135f85bd/users/79a7d0eb7feb48c782f78ba248ce2a9e/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:33.627407 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2159bb-bc70-435c-9ef9-7e1bcfb397b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72a0aec02ff04382b23a50fc135f85bd, user_id=79a7d0eb7feb48c782f78ba248ce2a9e, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:33.630194 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2159bb-bc70-435c-9ef9-7e1bcfb397b4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:33.706495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1262/2551] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:44:33 2022] PUT /identity/v3/projects/72a0aec02ff04382b23a50fc135f85bd/users/79a7d0eb7feb48c782f78ba248ce2a9e/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:44:33.710560 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53427e44-d0fc-4668-8e59-d017272c815b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:33.721180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53427e44-d0fc-4668-8e59-d017272c815b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:33.780758 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771e208-fa13-4485-a63c-24c2f0fe1abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:33.781445 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771e208-fa13-4485-a63c-24c2f0fe1abf admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:33.781725 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771e208-fa13-4485-a63c-24c2f0fe1abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:33.782001 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771e208-fa13-4485-a63c-24c2f0fe1abf admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:33.782714 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771e208-fa13-4485-a63c-24c2f0fe1abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:33.784307 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771e208-fa13-4485-a63c-24c2f0fe1abf admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:33.794729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1290/2552] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:44:33.799040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2159bb-bc70-435c-9ef9-7e1bcfb397b4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:33.809288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2159bb-bc70-435c-9ef9-7e1bcfb397b4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:33.813884 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48107e7-7db3-44df-a6d8-60a5c265c092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:33.814756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48107e7-7db3-44df-a6d8-60a5c265c092 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:33.815032 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48107e7-7db3-44df-a6d8-60a5c265c092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:33.815303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48107e7-7db3-44df-a6d8-60a5c265c092 admin admin] PATH_INFO: `/v3/projects/72a0aec02ff04382b23a50fc135f85bd/users/79a7d0eb7feb48c782f78ba248ce2a9e/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:33.817290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48107e7-7db3-44df-a6d8-60a5c265c092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72a0aec02ff04382b23a50fc135f85bd, user_id=79a7d0eb7feb48c782f78ba248ce2a9e, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:33.819926 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48107e7-7db3-44df-a6d8-60a5c265c092 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:33.830415 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1263/2553] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:44:33 2022] PUT /identity/v3/projects/72a0aec02ff04382b23a50fc135f85bd/users/79a7d0eb7feb48c782f78ba248ce2a9e/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:44:35.364039 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df272b-73c9-485e-9940-3c7436219a43 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:35.364570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df272b-73c9-485e-9940-3c7436219a43 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:35.364570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df272b-73c9-485e-9940-3c7436219a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:35.418755 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-08df272b-73c9-485e-9940-3c7436219a43 None None] MFA Rules not processed for user `79a7d0eb7feb48c782f78ba248ce2a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:35.419995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-08df272b-73c9-485e-9940-3c7436219a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:35.573854 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1291/2554] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:44:35 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:35.598353 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48107e7-7db3-44df-a6d8-60a5c265c092 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:35.600883 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e48107e7-7db3-44df-a6d8-60a5c265c092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:35.671904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:35.695416 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48107e7-7db3-44df-a6d8-60a5c265c092 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:35.705457 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87652373-822f-493b-a887-2b8e228b6589 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:35.706128 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87652373-822f-493b-a887-2b8e228b6589 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:35.706405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87652373-822f-493b-a887-2b8e228b6589 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:35.706677 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87652373-822f-493b-a887-2b8e228b6589 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:35.814627 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87652373-822f-493b-a887-2b8e228b6589 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:35.816088 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87652373-822f-493b-a887-2b8e228b6589 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:35.826482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1264/2555] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:44:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:44:37.593785 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df272b-73c9-485e-9940-3c7436219a43 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:37.603465 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df272b-73c9-485e-9940-3c7436219a43 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:37.607865 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a25fe-0a2c-492e-9988-409a1b947714 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:37.608619 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a25fe-0a2c-492e-9988-409a1b947714 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:37.608902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a25fe-0a2c-492e-9988-409a1b947714 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:37.609211 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a25fe-0a2c-492e-9988-409a1b947714 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:37.618428 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a25fe-0a2c-492e-9988-409a1b947714 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:37.619850 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a25fe-0a2c-492e-9988-409a1b947714 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:37.629914 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1292/2556] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:44:37 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:44:37.882127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87652373-822f-493b-a887-2b8e228b6589 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:37.940738 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87652373-822f-493b-a887-2b8e228b6589 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:37.944630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad685afd-d54b-49ed-8b54-685fcf49a747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:37.946631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad685afd-d54b-49ed-8b54-685fcf49a747 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:37.946631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad685afd-d54b-49ed-8b54-685fcf49a747 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:37.946631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad685afd-d54b-49ed-8b54-685fcf49a747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:37.947837 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ad685afd-d54b-49ed-8b54-685fcf49a747 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:37.959097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad685afd-d54b-49ed-8b54-685fcf49a747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:37.960965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad685afd-d54b-49ed-8b54-685fcf49a747 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:37.970928 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1265/2557] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:44:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:44:38.285821 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4440a32-9f4b-4bb6-97bc-1418b196a4f9 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:38.286205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4440a32-9f4b-4bb6-97bc-1418b196a4f9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:38.286572 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4440a32-9f4b-4bb6-97bc-1418b196a4f9 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:38.288277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1293/2558] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jun 7 08:44:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:44:38.296536 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec5250-2163-43c7-8a71-ec1ddf22598e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:38.296863 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec5250-2163-43c7-8a71-ec1ddf22598e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:38.297151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec5250-2163-43c7-8a71-ec1ddf22598e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:38.315239 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-deec5250-2163-43c7-8a71-ec1ddf22598e None None] MFA Rules not processed for user `ee42568b059b49a198ed2c01d5efdafd`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:38.320033 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-deec5250-2163-43c7-8a71-ec1ddf22598e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:38.385004 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1266/2559] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jun 7 08:44:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:38.397893 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4440a32-9f4b-4bb6-97bc-1418b196a4f9 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:38.399763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f4440a32-9f4b-4bb6-97bc-1418b196a4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:38.459628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4440a32-9f4b-4bb6-97bc-1418b196a4f9 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:38.463219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8f656-f459-45a1-925d-7adb527dabc9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4d9b5e3826049f2800f1b9424977d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:38.463867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8f656-f459-45a1-925d-7adb527dabc9 service glance] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:38.464157 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8f656-f459-45a1-925d-7adb527dabc9 service glance] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:38.464428 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8f656-f459-45a1-925d-7adb527dabc9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:38.475727 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8f656-f459-45a1-925d-7adb527dabc9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:38.477198 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8f656-f459-45a1-925d-7adb527dabc9 service glance] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:38.489072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1294/2560] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:44:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:44:38.676815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2073aa-84f1-4e1b-a117-181dc99f26f5 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:38.677143 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2073aa-84f1-4e1b-a117-181dc99f26f5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:38.677394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2073aa-84f1-4e1b-a117-181dc99f26f5 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:38.678938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1267/2561] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jun 7 08:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:44:38.684389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4de12-3f4c-4d40-b932-ef9cf4d460f0 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:38.684865 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4de12-3f4c-4d40-b932-ef9cf4d460f0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:38.684982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4de12-3f4c-4d40-b932-ef9cf4d460f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:38.703417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c0b4de12-3f4c-4d40-b932-ef9cf4d460f0 None None] MFA Rules not processed for user `4a87f8bb06ee4a3ba7017422f4f1943e`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:38.704543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c0b4de12-3f4c-4d40-b932-ef9cf4d460f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:38.711012 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1295/2562] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jun 7 08:44:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:38.722460 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2073aa-84f1-4e1b-a117-181dc99f26f5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:38.724095 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fe2073aa-84f1-4e1b-a117-181dc99f26f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:38.737051 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2073aa-84f1-4e1b-a117-181dc99f26f5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:38.740430 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b108a-1e4b-4e2e-a27c-6b494b8b1061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:38.741081 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b108a-1e4b-4e2e-a27c-6b494b8b1061 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:38.741360 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b108a-1e4b-4e2e-a27c-6b494b8b1061 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:38.741631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b108a-1e4b-4e2e-a27c-6b494b8b1061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:38.749110 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b108a-1e4b-4e2e-a27c-6b494b8b1061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:38.750541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b108a-1e4b-4e2e-a27c-6b494b8b1061 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:38.758134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1268/2563] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:44:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:44:38.840483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60e57b-6ea1-4832-aa63-ac7a6c2a5659 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:38.840778 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60e57b-6ea1-4832-aa63-ac7a6c2a5659 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:38.841092 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60e57b-6ea1-4832-aa63-ac7a6c2a5659 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:38.842461 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1296/2564] 10.222.0.17 () {54 vars in 921 bytes} [Tue Jun 7 08:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 08:44:38.847725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567d7dd-a98c-497f-993e-532d611095ff None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:38.848010 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567d7dd-a98c-497f-993e-532d611095ff None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:38.848283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567d7dd-a98c-497f-993e-532d611095ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:38.866077 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8567d7dd-a98c-497f-993e-532d611095ff None None] MFA Rules not processed for user `ee42568b059b49a198ed2c01d5efdafd`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:38.867267 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8567d7dd-a98c-497f-993e-532d611095ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:38.879581 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1269/2565] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Jun 7 08:44:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:38.890631 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60e57b-6ea1-4832-aa63-ac7a6c2a5659 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:38.897227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:38.901586 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60e57b-6ea1-4832-aa63-ac7a6c2a5659 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:38.905398 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef904e2-54ab-45eb-bba5-726e3375dff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:38.906031 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef904e2-54ab-45eb-bba5-726e3375dff4 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:38.906322 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef904e2-54ab-45eb-bba5-726e3375dff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:38.906598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef904e2-54ab-45eb-bba5-726e3375dff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:38.916933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef904e2-54ab-45eb-bba5-726e3375dff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:38.918393 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef904e2-54ab-45eb-bba5-726e3375dff4 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:38.929167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1297/2566] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:44:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:44:41.963180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5161d0d-6b75-40a3-8374-6a5e93ac6b89 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:41.963704 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5161d0d-6b75-40a3-8374-6a5e93ac6b89 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:41.963825 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5161d0d-6b75-40a3-8374-6a5e93ac6b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:41.971083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:41.981677 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a5161d0d-6b75-40a3-8374-6a5e93ac6b89 None None] MFA Rules not processed for user `bbf0ce396885472bb5d225aa009043f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:41.982873 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a5161d0d-6b75-40a3-8374-6a5e93ac6b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:42.036939 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1270/2567] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jun 7 08:44:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:42.049850 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef904e2-54ab-45eb-bba5-726e3375dff4 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:42.051237 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8ef904e2-54ab-45eb-bba5-726e3375dff4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:42.126836 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef904e2-54ab-45eb-bba5-726e3375dff4 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:42.131988 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf84fd-20c3-4c7a-9d8c-be6647c0db26 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334dc47ae2e643ccb44d25c6b5906f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:42.132539 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf84fd-20c3-4c7a-9d8c-be6647c0db26 service swift] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:42.132819 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf84fd-20c3-4c7a-9d8c-be6647c0db26 service swift] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:42.133117 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf84fd-20c3-4c7a-9d8c-be6647c0db26 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:42.143562 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf84fd-20c3-4c7a-9d8c-be6647c0db26 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:42.146427 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf84fd-20c3-4c7a-9d8c-be6647c0db26 service swift] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:42.154320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1298/2568] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jun 7 08:44:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:44:52.030276 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=23 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:52.150317 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=36 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:54.007343 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5161d0d-6b75-40a3-8374-6a5e93ac6b89 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:54.009647 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a5161d0d-6b75-40a3-8374-6a5e93ac6b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:54.115202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5161d0d-6b75-40a3-8374-6a5e93ac6b89 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:54.123659 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017a9fc-68dc-444b-add1-9c4ee1d62e2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc0d7bd7e4547f4a8d90ffc2c2e1d03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:54.124462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017a9fc-68dc-444b-add1-9c4ee1d62e2d service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:54.124911 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017a9fc-68dc-444b-add1-9c4ee1d62e2d service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:54.125354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017a9fc-68dc-444b-add1-9c4ee1d62e2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:54.127936 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7017a9fc-68dc-444b-add1-9c4ee1d62e2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:54.141614 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7017a9fc-68dc-444b-add1-9c4ee1d62e2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:54.143208 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7017a9fc-68dc-444b-add1-9c4ee1d62e2d service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:54.155821 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1271/2569] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:44:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:44:54.852459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf84fd-20c3-4c7a-9d8c-be6647c0db26 service swift] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:54.918135 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf84fd-20c3-4c7a-9d8c-be6647c0db26 service swift] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:54.923444 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45636a3-0f55-4135-bb4c-a7b38046a61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:54.924354 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45636a3-0f55-4135-bb4c-a7b38046a61b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:54.924678 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45636a3-0f55-4135-bb4c-a7b38046a61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:54.924984 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45636a3-0f55-4135-bb4c-a7b38046a61b admin admin] PATH_INFO: `/v3/users/a18749b8d6884acdbb7f6b7d816bb05d` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:54.926275 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45636a3-0f55-4135-bb4c-a7b38046a61b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a18749b8d6884acdbb7f6b7d816bb05d)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:54.928207 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45636a3-0f55-4135-bb4c-a7b38046a61b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:55.159551 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b45636a3-0f55-4135-bb4c-a7b38046a61b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a18749b8d6884acdbb7f6b7d816bb05d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:55.160281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b45636a3-0f55-4135-bb4c-a7b38046a61b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a18749b8d6884acdbb7f6b7d816bb05d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:55.167108 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b45636a3-0f55-4135-bb4c-a7b38046a61b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a18749b8d6884acdbb7f6b7d816bb05d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:55.167798 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b45636a3-0f55-4135-bb4c-a7b38046a61b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a18749b8d6884acdbb7f6b7d816bb05d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:55.252767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b45636a3-0f55-4135-bb4c-a7b38046a61b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a18749b8d6884acdbb7f6b7d816bb05d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:55.273888 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1299/2570] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:44:54 2022] DELETE /identity/v3/users/a18749b8d6884acdbb7f6b7d816bb05d => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:44:55.592967 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017a9fc-68dc-444b-add1-9c4ee1d62e2d service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:55.603721 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017a9fc-68dc-444b-add1-9c4ee1d62e2d service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:55.681317 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530a3d9-b43e-46c0-b0df-9e545f8d0a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:55.682317 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530a3d9-b43e-46c0-b0df-9e545f8d0a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:55.682550 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530a3d9-b43e-46c0-b0df-9e545f8d0a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:55.683226 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530a3d9-b43e-46c0-b0df-9e545f8d0a1a admin admin] PATH_INFO: `/v3/projects/e14f44aca77049c993e39c8373aaeaaa` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:55.684096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530a3d9-b43e-46c0-b0df-9e545f8d0a1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e14f44aca77049c993e39c8373aaeaaa)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:55.686217 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530a3d9-b43e-46c0-b0df-9e545f8d0a1a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:55.818299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d530a3d9-b43e-46c0-b0df-9e545f8d0a1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e14f44aca77049c993e39c8373aaeaaa', 'request_id': 'req-d530a3d9-b43e-46c0-b0df-9e545f8d0a1a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:55.831676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d530a3d9-b43e-46c0-b0df-9e545f8d0a1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e14f44aca77049c993e39c8373aaeaaa', 'request_id': 'req-d530a3d9-b43e-46c0-b0df-9e545f8d0a1a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:55.842934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d530a3d9-b43e-46c0-b0df-9e545f8d0a1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e14f44aca77049c993e39c8373aaeaaa', 'request_id': 'req-d530a3d9-b43e-46c0-b0df-9e545f8d0a1a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:55.938619 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d530a3d9-b43e-46c0-b0df-9e545f8d0a1a admin admin] The token cache is being invalidate because project e14f44aca77049c993e39c8373aaeaaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:44:55.938902 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d530a3d9-b43e-46c0-b0df-9e545f8d0a1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:55.939556 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d530a3d9-b43e-46c0-b0df-9e545f8d0a1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:55.942337 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1272/2571] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:44:55 2022] DELETE /identity/v3/projects/e14f44aca77049c993e39c8373aaeaaa => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:44:55.945793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45636a3-0f55-4135-bb4c-a7b38046a61b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:55.947320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b45636a3-0f55-4135-bb4c-a7b38046a61b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:55.960879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45636a3-0f55-4135-bb4c-a7b38046a61b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:56.025676 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f3b46-e927-4ab3-860a-1c52b4e6ff95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:56.026638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f3b46-e927-4ab3-860a-1c52b4e6ff95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:56.028982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f3b46-e927-4ab3-860a-1c52b4e6ff95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:56.028982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f3b46-e927-4ab3-860a-1c52b4e6ff95 admin admin] PATH_INFO: `/v3/users/79a7d0eb7feb48c782f78ba248ce2a9e` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:56.029814 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f3b46-e927-4ab3-860a-1c52b4e6ff95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79a7d0eb7feb48c782f78ba248ce2a9e)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:56.032016 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f3b46-e927-4ab3-860a-1c52b4e6ff95 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:56.058077 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:56.243946 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-085f3b46-e927-4ab3-860a-1c52b4e6ff95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79a7d0eb7feb48c782f78ba248ce2a9e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:56.247102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-085f3b46-e927-4ab3-860a-1c52b4e6ff95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79a7d0eb7feb48c782f78ba248ce2a9e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:56.252065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-085f3b46-e927-4ab3-860a-1c52b4e6ff95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79a7d0eb7feb48c782f78ba248ce2a9e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:56.252769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-085f3b46-e927-4ab3-860a-1c52b4e6ff95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79a7d0eb7feb48c782f78ba248ce2a9e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:56.332938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-085f3b46-e927-4ab3-860a-1c52b4e6ff95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79a7d0eb7feb48c782f78ba248ce2a9e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:56.351703 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1300/2572] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:44:55 2022] DELETE /identity/v3/users/79a7d0eb7feb48c782f78ba248ce2a9e => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:44:56.724900 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530a3d9-b43e-46c0-b0df-9e545f8d0a1a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:56.726406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d530a3d9-b43e-46c0-b0df-9e545f8d0a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:56.743012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530a3d9-b43e-46c0-b0df-9e545f8d0a1a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:56.773276 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:56.819486 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a7d76-d063-463e-a1f9-231cf0eaa59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:56.819486 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a7d76-d063-463e-a1f9-231cf0eaa59b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:56.819486 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a7d76-d063-463e-a1f9-231cf0eaa59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:56.819486 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a7d76-d063-463e-a1f9-231cf0eaa59b admin admin] PATH_INFO: `/v3/projects/72a0aec02ff04382b23a50fc135f85bd` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:56.819866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a7d76-d063-463e-a1f9-231cf0eaa59b admin admin] RBAC: Authorizing `identity:delete_project(project_id=72a0aec02ff04382b23a50fc135f85bd)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:56.819866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a7d76-d063-463e-a1f9-231cf0eaa59b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:56.957837 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6d2a7d76-d063-463e-a1f9-231cf0eaa59b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72a0aec02ff04382b23a50fc135f85bd', 'request_id': 'req-6d2a7d76-d063-463e-a1f9-231cf0eaa59b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:56.963093 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6d2a7d76-d063-463e-a1f9-231cf0eaa59b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72a0aec02ff04382b23a50fc135f85bd', 'request_id': 'req-6d2a7d76-d063-463e-a1f9-231cf0eaa59b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:56.969222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6d2a7d76-d063-463e-a1f9-231cf0eaa59b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72a0aec02ff04382b23a50fc135f85bd', 'request_id': 'req-6d2a7d76-d063-463e-a1f9-231cf0eaa59b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:57.026105 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6d2a7d76-d063-463e-a1f9-231cf0eaa59b admin admin] The token cache is being invalidate because project 72a0aec02ff04382b23a50fc135f85bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:44:57.026544 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6d2a7d76-d063-463e-a1f9-231cf0eaa59b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:57.027542 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6d2a7d76-d063-463e-a1f9-231cf0eaa59b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:44:57.034287 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1273/2573] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:44:56 2022] DELETE /identity/v3/projects/72a0aec02ff04382b23a50fc135f85bd => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:44:57.036270 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780a00b-41e6-4159-8f15-54f652f868a2 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:57.036834 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780a00b-41e6-4159-8f15-54f652f868a2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:57.036971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780a00b-41e6-4159-8f15-54f652f868a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:57.057628 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-5780a00b-41e6-4159-8f15-54f652f868a2 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:57.060850 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5780a00b-41e6-4159-8f15-54f652f868a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:57.141241 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1301/2574] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:57.146684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c79d0-68dc-40bc-aeeb-db6fd174c863 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:57.146962 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c79d0-68dc-40bc-aeeb-db6fd174c863 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:57.147235 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c79d0-68dc-40bc-aeeb-db6fd174c863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:57.166949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bf0c79d0-68dc-40bc-aeeb-db6fd174c863 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:57.168251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bf0c79d0-68dc-40bc-aeeb-db6fd174c863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:57.177447 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1274/2575] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:57.185991 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780a00b-41e6-4159-8f15-54f652f868a2 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:57.196635 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780a00b-41e6-4159-8f15-54f652f868a2 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:57.201467 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7430f-a549-4c47-9a85-b36f74d20573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:57.202132 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7430f-a549-4c47-9a85-b36f74d20573 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:57.202418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7430f-a549-4c47-9a85-b36f74d20573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:57.202695 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7430f-a549-4c47-9a85-b36f74d20573 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:57.203432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7430f-a549-4c47-9a85-b36f74d20573 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:44:57.203741 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7430f-a549-4c47-9a85-b36f74d20573 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:57.205387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7430f-a549-4c47-9a85-b36f74d20573 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:57.218801 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1302/2576] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:44:57.223152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c79d0-68dc-40bc-aeeb-db6fd174c863 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:57.233159 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c79d0-68dc-40bc-aeeb-db6fd174c863 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:57.238134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7562453e-f6e0-4024-bf08-561e7db27861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:57.238790 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562453e-f6e0-4024-bf08-561e7db27861 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:57.239091 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562453e-f6e0-4024-bf08-561e7db27861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:57.239366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562453e-f6e0-4024-bf08-561e7db27861 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:57.240067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7562453e-f6e0-4024-bf08-561e7db27861 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:57.242007 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7562453e-f6e0-4024-bf08-561e7db27861 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:57.346543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1275/2577] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:44:57 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:44:57.350683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7430f-a549-4c47-9a85-b36f74d20573 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:57.361234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7430f-a549-4c47-9a85-b36f74d20573 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:57.420502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608590f0-1d58-45d0-90a3-ece75899b2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:57.421204 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608590f0-1d58-45d0-90a3-ece75899b2ed admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:57.421485 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608590f0-1d58-45d0-90a3-ece75899b2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:57.421761 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608590f0-1d58-45d0-90a3-ece75899b2ed admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:57.422448 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608590f0-1d58-45d0-90a3-ece75899b2ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:57.424378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608590f0-1d58-45d0-90a3-ece75899b2ed admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:57.514856 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1303/2578] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:44:57 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:44:57.516587 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7562453e-f6e0-4024-bf08-561e7db27861 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:57.528932 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7562453e-f6e0-4024-bf08-561e7db27861 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:57.534082 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5704841-b43f-453f-b448-ae5ce85197b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:57.534747 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5704841-b43f-453f-b448-ae5ce85197b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:57.535033 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5704841-b43f-453f-b448-ae5ce85197b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:57.535301 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5704841-b43f-453f-b448-ae5ce85197b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:57.536032 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5704841-b43f-453f-b448-ae5ce85197b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:57.537767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5704841-b43f-453f-b448-ae5ce85197b0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:57.549296 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1276/2579] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:44:57.553243 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608590f0-1d58-45d0-90a3-ece75899b2ed admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:57.563748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608590f0-1d58-45d0-90a3-ece75899b2ed admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:57.568524 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a811f5-0fb4-49c8-a62e-5ce908ce0702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:57.569218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a811f5-0fb4-49c8-a62e-5ce908ce0702 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:57.569500 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a811f5-0fb4-49c8-a62e-5ce908ce0702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:57.569776 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a811f5-0fb4-49c8-a62e-5ce908ce0702 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:57.570530 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a811f5-0fb4-49c8-a62e-5ce908ce0702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:57.572123 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a811f5-0fb4-49c8-a62e-5ce908ce0702 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:57.584299 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1304/2580] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:44:57.587999 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5704841-b43f-453f-b448-ae5ce85197b0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:57.598800 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5704841-b43f-453f-b448-ae5ce85197b0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:57.603357 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ef6d3-f456-444a-aa79-ea66bb94a917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:57.604248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ef6d3-f456-444a-aa79-ea66bb94a917 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:57.604525 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ef6d3-f456-444a-aa79-ea66bb94a917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:57.604794 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ef6d3-f456-444a-aa79-ea66bb94a917 admin admin] PATH_INFO: `/v3/projects/ec5703dfcb5e4ff8826addb6ca478e51/users/895951de660840628ca5ec61d4f311cc/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:57.626089 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ef6d3-f456-444a-aa79-ea66bb94a917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec5703dfcb5e4ff8826addb6ca478e51, user_id=895951de660840628ca5ec61d4f311cc, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:57.628786 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ef6d3-f456-444a-aa79-ea66bb94a917 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:57.693991 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1277/2581] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:44:57 2022] PUT /identity/v3/projects/ec5703dfcb5e4ff8826addb6ca478e51/users/895951de660840628ca5ec61d4f311cc/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:44:57.704978 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5387530-319e-43c7-b113-b0b07fc84a48 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:57.705312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5387530-319e-43c7-b113-b0b07fc84a48 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:57.705596 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5387530-319e-43c7-b113-b0b07fc84a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:57.714109 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:57.735352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a5387530-319e-43c7-b113-b0b07fc84a48 None None] MFA Rules not processed for user `895951de660840628ca5ec61d4f311cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:57.737022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a5387530-319e-43c7-b113-b0b07fc84a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:57.818191 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1305/2582] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:44:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:44:57.829295 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ef6d3-f456-444a-aa79-ea66bb94a917 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:57.833311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6f8ef6d3-f456-444a-aa79-ea66bb94a917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:57.884842 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:44:57.891336 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ef6d3-f456-444a-aa79-ea66bb94a917 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:57.895906 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f22a0-a159-4854-8196-bcee825cb815 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:57.900300 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f22a0-a159-4854-8196-bcee825cb815 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:57.900300 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f22a0-a159-4854-8196-bcee825cb815 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:57.900300 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f22a0-a159-4854-8196-bcee825cb815 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:57.906872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f22a0-a159-4854-8196-bcee825cb815 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:57.908299 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f22a0-a159-4854-8196-bcee825cb815 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:57.917043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1278/2583] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:44:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:44:59.113107 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261271f4-a7ae-44f2-b99b-9ff7ceb0b50c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:59.113846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261271f4-a7ae-44f2-b99b-9ff7ceb0b50c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:59.113846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261271f4-a7ae-44f2-b99b-9ff7ceb0b50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:59.133506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-261271f4-a7ae-44f2-b99b-9ff7ceb0b50c None None] MFA Rules not processed for user `bbf0ce396885472bb5d225aa009043f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:59.136339 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-261271f4-a7ae-44f2-b99b-9ff7ceb0b50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:59.190395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1306/2584] 10.222.0.17 () {58 vars in 967 bytes} [Tue Jun 7 08:44:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:44:59.215028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f22a0-a159-4854-8196-bcee825cb815 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:44:59.217522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1f0f22a0-a159-4854-8196-bcee825cb815 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:59.290131 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f22a0-a159-4854-8196-bcee825cb815 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:44:59.296392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3b2dd-4e57-4d84-ac05-66e34424a7d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334dc47ae2e643ccb44d25c6b5906f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:44:59.297125 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3b2dd-4e57-4d84-ac05-66e34424a7d1 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:59.297445 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3b2dd-4e57-4d84-ac05-66e34424a7d1 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:59.297728 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3b2dd-4e57-4d84-ac05-66e34424a7d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:59.306580 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3b2dd-4e57-4d84-ac05-66e34424a7d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:44:59.308173 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3b2dd-4e57-4d84-ac05-66e34424a7d1 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:44:59.316743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1279/2585] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jun 7 08:44:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:44:59.611125 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80e5b9-3be7-47e8-a897-54b799706e86 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:44:59.611412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80e5b9-3be7-47e8-a897-54b799706e86 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:44:59.611688 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80e5b9-3be7-47e8-a897-54b799706e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:44:59.632006 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-af80e5b9-3be7-47e8-a897-54b799706e86 None None] MFA Rules not processed for user `895951de660840628ca5ec61d4f311cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:44:59.633306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-af80e5b9-3be7-47e8-a897-54b799706e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:44:59.640970 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1307/2586] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:44:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:45:02.210092 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3b2dd-4e57-4d84-ac05-66e34424a7d1 service swift] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:02.211981 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-95a3b2dd-4e57-4d84-ac05-66e34424a7d1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:02.278249 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3b2dd-4e57-4d84-ac05-66e34424a7d1 service swift] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:02.283844 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ed43c-e642-4889-a7cd-020155e72939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:02.284639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ed43c-e642-4889-a7cd-020155e72939 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:02.285898 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ed43c-e642-4889-a7cd-020155e72939 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:02.286339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ed43c-e642-4889-a7cd-020155e72939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:02.288455 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f31ed43c-e642-4889-a7cd-020155e72939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:02.361557 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31ed43c-e642-4889-a7cd-020155e72939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:02.362992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31ed43c-e642-4889-a7cd-020155e72939 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:02.372534 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1280/2587] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:45:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:45:09.104307 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80e5b9-3be7-47e8-a897-54b799706e86 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:09.105706 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-af80e5b9-3be7-47e8-a897-54b799706e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:09.117497 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80e5b9-3be7-47e8-a897-54b799706e86 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:09.120975 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0eb310-f21d-4a81-9d4f-645c3c17cdaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:09.122156 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0eb310-f21d-4a81-9d4f-645c3c17cdaf service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:09.122451 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0eb310-f21d-4a81-9d4f-645c3c17cdaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:09.122716 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0eb310-f21d-4a81-9d4f-645c3c17cdaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:09.124404 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1e0eb310-f21d-4a81-9d4f-645c3c17cdaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:09.189915 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0eb310-f21d-4a81-9d4f-645c3c17cdaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:09.191169 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0eb310-f21d-4a81-9d4f-645c3c17cdaf service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:09.201531 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1308/2588] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:45:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:45:12.368273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=87 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:45:13.642714 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ed43c-e642-4889-a7cd-020155e72939 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:13.644379 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f31ed43c-e642-4889-a7cd-020155e72939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:13.660350 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ed43c-e642-4889-a7cd-020155e72939 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:13.664940 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8084c9-16d3-4575-944c-9a02ff7ba4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:13.665839 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8084c9-16d3-4575-944c-9a02ff7ba4b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:13.666122 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8084c9-16d3-4575-944c-9a02ff7ba4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:13.666396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8084c9-16d3-4575-944c-9a02ff7ba4b2 admin admin] PATH_INFO: `/v3/users/4e5893f0909446ff82b5a72584ecae37` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:13.667625 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8084c9-16d3-4575-944c-9a02ff7ba4b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e5893f0909446ff82b5a72584ecae37)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:13.669720 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8084c9-16d3-4575-944c-9a02ff7ba4b2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:13.855211 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6c8084c9-16d3-4575-944c-9a02ff7ba4b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e5893f0909446ff82b5a72584ecae37'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:13.855911 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6c8084c9-16d3-4575-944c-9a02ff7ba4b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e5893f0909446ff82b5a72584ecae37'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:13.863092 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6c8084c9-16d3-4575-944c-9a02ff7ba4b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e5893f0909446ff82b5a72584ecae37'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:13.863794 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6c8084c9-16d3-4575-944c-9a02ff7ba4b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e5893f0909446ff82b5a72584ecae37'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:13.876378 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6c8084c9-16d3-4575-944c-9a02ff7ba4b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e5893f0909446ff82b5a72584ecae37'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:13.961316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1281/2589] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:45:13 2022] DELETE /identity/v3/users/4e5893f0909446ff82b5a72584ecae37 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:45:14.254581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0eb310-f21d-4a81-9d4f-645c3c17cdaf service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:14.264856 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0eb310-f21d-4a81-9d4f-645c3c17cdaf service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:14.304354 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:45:14.337024 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95831070-3ffb-4138-86ad-f4a61bb48dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:14.339028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95831070-3ffb-4138-86ad-f4a61bb48dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:14.339314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95831070-3ffb-4138-86ad-f4a61bb48dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:14.339593 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95831070-3ffb-4138-86ad-f4a61bb48dea admin admin] PATH_INFO: `/v3/projects/5daf862eb4de4ea992b35692f003a25e` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:14.341393 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95831070-3ffb-4138-86ad-f4a61bb48dea admin admin] RBAC: Authorizing `identity:delete_project(project_id=5daf862eb4de4ea992b35692f003a25e)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:14.343333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95831070-3ffb-4138-86ad-f4a61bb48dea admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:14.441889 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-95831070-3ffb-4138-86ad-f4a61bb48dea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5daf862eb4de4ea992b35692f003a25e', 'request_id': 'req-95831070-3ffb-4138-86ad-f4a61bb48dea'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:14.448335 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-95831070-3ffb-4138-86ad-f4a61bb48dea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5daf862eb4de4ea992b35692f003a25e', 'request_id': 'req-95831070-3ffb-4138-86ad-f4a61bb48dea'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:14.509763 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-95831070-3ffb-4138-86ad-f4a61bb48dea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5daf862eb4de4ea992b35692f003a25e', 'request_id': 'req-95831070-3ffb-4138-86ad-f4a61bb48dea'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:14.521206 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-95831070-3ffb-4138-86ad-f4a61bb48dea admin admin] The token cache is being invalidate because project 5daf862eb4de4ea992b35692f003a25e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:45:14.521573 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-95831070-3ffb-4138-86ad-f4a61bb48dea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:14.522314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-95831070-3ffb-4138-86ad-f4a61bb48dea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:14.523808 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1309/2590] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:45:14 2022] DELETE /identity/v3/projects/5daf862eb4de4ea992b35692f003a25e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:45:14.529806 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdfe8e-1ee4-4d4a-ad17-9278ff597abf None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:14.530162 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdfe8e-1ee4-4d4a-ad17-9278ff597abf None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:14.530504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdfe8e-1ee4-4d4a-ad17-9278ff597abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:14.551234 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-44cdfe8e-1ee4-4d4a-ad17-9278ff597abf None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:45:14.552568 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-44cdfe8e-1ee4-4d4a-ad17-9278ff597abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:14.612148 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1282/2591] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:45:14.620116 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff3b88-4a38-4e02-8080-1254d5976e1e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:14.620410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff3b88-4a38-4e02-8080-1254d5976e1e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:14.620693 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff3b88-4a38-4e02-8080-1254d5976e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:14.638959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-2dff3b88-4a38-4e02-8080-1254d5976e1e None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:45:14.640195 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2dff3b88-4a38-4e02-8080-1254d5976e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:14.648237 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1310/2592] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:45:14.652495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cdfe8e-1ee4-4d4a-ad17-9278ff597abf None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:14.663108 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cdfe8e-1ee4-4d4a-ad17-9278ff597abf None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:14.668272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000df66-9a16-48ae-9a12-42583d855618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:14.669568 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000df66-9a16-48ae-9a12-42583d855618 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:14.669874 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000df66-9a16-48ae-9a12-42583d855618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:14.670143 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000df66-9a16-48ae-9a12-42583d855618 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:14.670867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0000df66-9a16-48ae-9a12-42583d855618 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:45:14.671172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0000df66-9a16-48ae-9a12-42583d855618 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:14.672776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0000df66-9a16-48ae-9a12-42583d855618 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:14.685164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1283/2593] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:45:14.689620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff3b88-4a38-4e02-8080-1254d5976e1e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:14.699152 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff3b88-4a38-4e02-8080-1254d5976e1e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:14.704381 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fbd23-7b00-479c-a52e-aa0088375079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:14.705074 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fbd23-7b00-479c-a52e-aa0088375079 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:14.705362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fbd23-7b00-479c-a52e-aa0088375079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:14.705640 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fbd23-7b00-479c-a52e-aa0088375079 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:14.706451 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fbd23-7b00-479c-a52e-aa0088375079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:14.708395 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fbd23-7b00-479c-a52e-aa0088375079 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:14.791220 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1311/2594] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:45:14 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:45:14.795421 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000df66-9a16-48ae-9a12-42583d855618 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:14.807997 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000df66-9a16-48ae-9a12-42583d855618 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:14.865927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b2f54-9a2f-42b0-9d6e-38fcb80c7a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:14.866606 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b2f54-9a2f-42b0-9d6e-38fcb80c7a64 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:14.866890 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b2f54-9a2f-42b0-9d6e-38fcb80c7a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:14.867167 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b2f54-9a2f-42b0-9d6e-38fcb80c7a64 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:14.867874 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b2f54-9a2f-42b0-9d6e-38fcb80c7a64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:14.869985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b2f54-9a2f-42b0-9d6e-38fcb80c7a64 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:14.876880 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:45:14.973039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1284/2595] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:45:14 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:45:14.977010 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fbd23-7b00-479c-a52e-aa0088375079 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:14.987571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fbd23-7b00-479c-a52e-aa0088375079 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:14.992259 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2c1d1-fe47-48b2-9aa8-a85d1a0b3d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:14.992938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2c1d1-fe47-48b2-9aa8-a85d1a0b3d62 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:14.993243 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2c1d1-fe47-48b2-9aa8-a85d1a0b3d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:14.993520 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2c1d1-fe47-48b2-9aa8-a85d1a0b3d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:14.997156 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2c1d1-fe47-48b2-9aa8-a85d1a0b3d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:14.997156 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2c1d1-fe47-48b2-9aa8-a85d1a0b3d62 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:15.006478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1312/2596] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:45:15.011062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b2f54-9a2f-42b0-9d6e-38fcb80c7a64 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:15.021582 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b2f54-9a2f-42b0-9d6e-38fcb80c7a64 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:15.029356 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13986b-8841-4bf9-9d8c-f2edba62376d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:15.029356 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13986b-8841-4bf9-9d8c-f2edba62376d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:15.029356 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13986b-8841-4bf9-9d8c-f2edba62376d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:15.029356 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13986b-8841-4bf9-9d8c-f2edba62376d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:15.029716 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13986b-8841-4bf9-9d8c-f2edba62376d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:15.030658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13986b-8841-4bf9-9d8c-f2edba62376d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:15.041251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1285/2597] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:45:15.045541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2c1d1-fe47-48b2-9aa8-a85d1a0b3d62 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:15.055679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2c1d1-fe47-48b2-9aa8-a85d1a0b3d62 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:15.061175 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d2294-ddbd-413d-96cc-a1a457b79ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:15.062053 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d2294-ddbd-413d-96cc-a1a457b79ace admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:15.062337 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d2294-ddbd-413d-96cc-a1a457b79ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:15.062605 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d2294-ddbd-413d-96cc-a1a457b79ace admin admin] PATH_INFO: `/v3/projects/0899cad1b7c64a0199b3466d4c7cfefb/users/52ddf59fe764404eb1c6ee745e21d352/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:15.077734 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d2294-ddbd-413d-96cc-a1a457b79ace admin admin] RBAC: Authorizing `identity:create_grant(project_id=0899cad1b7c64a0199b3466d4c7cfefb, user_id=52ddf59fe764404eb1c6ee745e21d352, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:15.080417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d2294-ddbd-413d-96cc-a1a457b79ace admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:15.127310 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1313/2598] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:45:15 2022] PUT /identity/v3/projects/0899cad1b7c64a0199b3466d4c7cfefb/users/52ddf59fe764404eb1c6ee745e21d352/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:45:15.132409 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555196a1-11b7-42c6-89b6-9c1153608a52 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:15.132938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555196a1-11b7-42c6-89b6-9c1153608a52 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:15.133406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555196a1-11b7-42c6-89b6-9c1153608a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:15.151709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-555196a1-11b7-42c6-89b6-9c1153608a52 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:45:15.152907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-555196a1-11b7-42c6-89b6-9c1153608a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:15.214724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1286/2599] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:45:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:45:15.221298 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d2294-ddbd-413d-96cc-a1a457b79ace admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:15.223002 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-195d2294-ddbd-413d-96cc-a1a457b79ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:15.278846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d2294-ddbd-413d-96cc-a1a457b79ace admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:15.282868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099b985-c938-46ce-ae94-44a2f58013dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:15.283514 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099b985-c938-46ce-ae94-44a2f58013dd service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:15.283797 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099b985-c938-46ce-ae94-44a2f58013dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:15.284069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099b985-c938-46ce-ae94-44a2f58013dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:15.291059 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:45:15.293610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099b985-c938-46ce-ae94-44a2f58013dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:15.295040 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099b985-c938-46ce-ae94-44a2f58013dd service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:15.305149 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1314/2600] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:45:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:45:18.520028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001ab6e-3f99-4676-8435-4bae9a2635d0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:18.521124 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001ab6e-3f99-4676-8435-4bae9a2635d0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:18.521639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001ab6e-3f99-4676-8435-4bae9a2635d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:18.557591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0001ab6e-3f99-4676-8435-4bae9a2635d0 None None] MFA Rules not processed for user `bbf0ce396885472bb5d225aa009043f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:45:18.559042 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0001ab6e-3f99-4676-8435-4bae9a2635d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:18.633484 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1287/2601] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jun 7 08:45:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:45:18.650117 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099b985-c938-46ce-ae94-44a2f58013dd service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:18.651873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2099b985-c938-46ce-ae94-44a2f58013dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:18.745147 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099b985-c938-46ce-ae94-44a2f58013dd service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:18.751072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948b8d4-004d-4859-9229-30e03840a6fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334dc47ae2e643ccb44d25c6b5906f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:18.751940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948b8d4-004d-4859-9229-30e03840a6fd service swift] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:18.752215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948b8d4-004d-4859-9229-30e03840a6fd service swift] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:18.752646 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948b8d4-004d-4859-9229-30e03840a6fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:18.763158 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948b8d4-004d-4859-9229-30e03840a6fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:18.764757 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948b8d4-004d-4859-9229-30e03840a6fd service swift] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:18.773567 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1315/2602] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jun 7 08:45:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:45:19.841031 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dbb27-7069-4de9-ae59-61ff4b573c6e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:19.841578 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dbb27-7069-4de9-ae59-61ff4b573c6e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:19.841873 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dbb27-7069-4de9-ae59-61ff4b573c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:19.863757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-172dbb27-7069-4de9-ae59-61ff4b573c6e None None] MFA Rules not processed for user `bbf0ce396885472bb5d225aa009043f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:45:19.865789 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-172dbb27-7069-4de9-ae59-61ff4b573c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:19.876569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1288/2603] 10.222.0.17 () {58 vars in 967 bytes} [Tue Jun 7 08:45:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:45:19.891433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948b8d4-004d-4859-9229-30e03840a6fd service swift] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:19.904188 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948b8d4-004d-4859-9229-30e03840a6fd service swift] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:19.909874 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0613d-cbe7-45b5-8e92-93f0e66ee0df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334dc47ae2e643ccb44d25c6b5906f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:19.910630 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0613d-cbe7-45b5-8e92-93f0e66ee0df service swift] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:19.911136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0613d-cbe7-45b5-8e92-93f0e66ee0df service swift] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:19.911483 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0613d-cbe7-45b5-8e92-93f0e66ee0df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:19.920912 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0613d-cbe7-45b5-8e92-93f0e66ee0df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:19.923199 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0613d-cbe7-45b5-8e92-93f0e66ee0df service swift] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:19.931851 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1316/2604] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jun 7 08:45:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:45:20.530237 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9614ba5-ec60-431f-93f0-83fe4099e98b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:20.530738 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9614ba5-ec60-431f-93f0-83fe4099e98b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:20.531088 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9614ba5-ec60-431f-93f0-83fe4099e98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:20.562126 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c9614ba5-ec60-431f-93f0-83fe4099e98b None None] MFA Rules not processed for user `52ddf59fe764404eb1c6ee745e21d352`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:45:20.563331 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c9614ba5-ec60-431f-93f0-83fe4099e98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:20.588456 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7517abc-180b-419a-b88e-d2b0761d1d12 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:20.588744 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7517abc-180b-419a-b88e-d2b0761d1d12 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:20.589334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7517abc-180b-419a-b88e-d2b0761d1d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:20.610020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d7517abc-180b-419a-b88e-d2b0761d1d12 None None] MFA Rules not processed for user `bbf0ce396885472bb5d225aa009043f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:45:20.611592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d7517abc-180b-419a-b88e-d2b0761d1d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:20.619734 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1317/2605] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Jun 7 08:45:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:45:20.633234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7517abc-180b-419a-b88e-d2b0761d1d12 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:20.654772 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7517abc-180b-419a-b88e-d2b0761d1d12 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:20.663120 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1289/2606] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:45:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:45:20.671246 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5dcca3-a04a-4fbe-93b1-a0147399dee1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334dc47ae2e643ccb44d25c6b5906f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:20.672146 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dcca3-a04a-4fbe-93b1-a0147399dee1 service swift] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:20.672602 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dcca3-a04a-4fbe-93b1-a0147399dee1 service swift] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:20.672882 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dcca3-a04a-4fbe-93b1-a0147399dee1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:20.678289 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9614ba5-ec60-431f-93f0-83fe4099e98b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:20.679674 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c9614ba5-ec60-431f-93f0-83fe4099e98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:20.683229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5dcca3-a04a-4fbe-93b1-a0147399dee1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:20.685416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5dcca3-a04a-4fbe-93b1-a0147399dee1 service swift] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:20.690346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:45:20.695205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1318/2607] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jun 7 08:45:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 08:45:20.766068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9614ba5-ec60-431f-93f0-83fe4099e98b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:20.773213 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a67fa0-cb5e-4af8-8ebf-4c082aba4d78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:20.774020 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a67fa0-cb5e-4af8-8ebf-4c082aba4d78 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:20.774314 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a67fa0-cb5e-4af8-8ebf-4c082aba4d78 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:20.774597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a67fa0-cb5e-4af8-8ebf-4c082aba4d78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:20.784271 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a67fa0-cb5e-4af8-8ebf-4c082aba4d78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:20.785993 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a67fa0-cb5e-4af8-8ebf-4c082aba4d78 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:20.796541 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1290/2608] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:45:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:45:21.430776 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9634f-665d-4b0a-b0d9-07d1f5966c1d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:21.431215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9634f-665d-4b0a-b0d9-07d1f5966c1d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:21.431360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9634f-665d-4b0a-b0d9-07d1f5966c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:21.452108 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-4ea9634f-665d-4b0a-b0d9-07d1f5966c1d None None] MFA Rules not processed for user `bbf0ce396885472bb5d225aa009043f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:45:21.455714 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4ea9634f-665d-4b0a-b0d9-07d1f5966c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:21.461285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1319/2609] 10.222.0.17 () {58 vars in 967 bytes} [Tue Jun 7 08:45:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:45:21.474645 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a67fa0-cb5e-4af8-8ebf-4c082aba4d78 service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:21.486745 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a67fa0-cb5e-4af8-8ebf-4c082aba4d78 service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:21.495041 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca168231-a6bb-448b-8746-69c7ec280a5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334dc47ae2e643ccb44d25c6b5906f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:21.495041 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca168231-a6bb-448b-8746-69c7ec280a5a service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:21.495041 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca168231-a6bb-448b-8746-69c7ec280a5a service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:21.495041 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca168231-a6bb-448b-8746-69c7ec280a5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:21.503543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca168231-a6bb-448b-8746-69c7ec280a5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:21.505424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca168231-a6bb-448b-8746-69c7ec280a5a service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:21.515036 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1291/2610] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Jun 7 08:45:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 08:45:23.586020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f28b74-1d68-45c3-aabe-aa4d386d1d69 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:23.589482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f28b74-1d68-45c3-aabe-aa4d386d1d69 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:23.589482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f28b74-1d68-45c3-aabe-aa4d386d1d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:23.606415 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c5f28b74-1d68-45c3-aabe-aa4d386d1d69 None None] MFA Rules not processed for user `52ddf59fe764404eb1c6ee745e21d352`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:45:23.607949 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c5f28b74-1d68-45c3-aabe-aa4d386d1d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:23.615827 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1320/2611] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:45:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:45:24.557472 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca168231-a6bb-448b-8746-69c7ec280a5a service swift] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:24.559866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ca168231-a6bb-448b-8746-69c7ec280a5a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:24.577684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca168231-a6bb-448b-8746-69c7ec280a5a service swift] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:24.583135 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e299f0-c35e-4049-b17f-609a90c941b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:24.583724 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e299f0-c35e-4049-b17f-609a90c941b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:24.584006 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e299f0-c35e-4049-b17f-609a90c941b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:24.584281 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e299f0-c35e-4049-b17f-609a90c941b7 admin admin] PATH_INFO: `/v3/users/895951de660840628ca5ec61d4f311cc` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:24.586313 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e299f0-c35e-4049-b17f-609a90c941b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=895951de660840628ca5ec61d4f311cc)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:24.588225 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e299f0-c35e-4049-b17f-609a90c941b7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:24.783504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-33e299f0-c35e-4049-b17f-609a90c941b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '895951de660840628ca5ec61d4f311cc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:24.784484 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-33e299f0-c35e-4049-b17f-609a90c941b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '895951de660840628ca5ec61d4f311cc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:24.792532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-33e299f0-c35e-4049-b17f-609a90c941b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '895951de660840628ca5ec61d4f311cc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:24.800118 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-33e299f0-c35e-4049-b17f-609a90c941b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '895951de660840628ca5ec61d4f311cc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:24.807528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-33e299f0-c35e-4049-b17f-609a90c941b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '895951de660840628ca5ec61d4f311cc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:24.904573 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1292/2612] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:45:24 2022] DELETE /identity/v3/users/895951de660840628ca5ec61d4f311cc => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:45:24.909061 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998df2c2-a718-4e8e-b6cb-9b05f2f13db0 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:24.909525 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998df2c2-a718-4e8e-b6cb-9b05f2f13db0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:24.909647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998df2c2-a718-4e8e-b6cb-9b05f2f13db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:24.927605 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-998df2c2-a718-4e8e-b6cb-9b05f2f13db0 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:45:24.928800 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-998df2c2-a718-4e8e-b6cb-9b05f2f13db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:24.992528 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1321/2613] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:45:25.002382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e299f0-c35e-4049-b17f-609a90c941b7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:25.007124 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-33e299f0-c35e-4049-b17f-609a90c941b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:25.062274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e299f0-c35e-4049-b17f-609a90c941b7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:25.066193 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32441c-a14e-4beb-8750-3f54dedf5a50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:25.066851 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32441c-a14e-4beb-8750-3f54dedf5a50 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:25.067135 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32441c-a14e-4beb-8750-3f54dedf5a50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:25.067411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32441c-a14e-4beb-8750-3f54dedf5a50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:25.073691 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:45:25.077452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32441c-a14e-4beb-8750-3f54dedf5a50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:25.078899 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32441c-a14e-4beb-8750-3f54dedf5a50 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:25.089327 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1293/2614] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:45:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:45:25.322209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998df2c2-a718-4e8e-b6cb-9b05f2f13db0 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:25.323652 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-998df2c2-a718-4e8e-b6cb-9b05f2f13db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:25.336259 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998df2c2-a718-4e8e-b6cb-9b05f2f13db0 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:25.341676 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e537e-fd13-41bd-8b53-bef9cf661b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:25.342546 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e537e-fd13-41bd-8b53-bef9cf661b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:25.342828 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e537e-fd13-41bd-8b53-bef9cf661b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:25.343104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e537e-fd13-41bd-8b53-bef9cf661b4b admin admin] PATH_INFO: `/v3/projects/ec5703dfcb5e4ff8826addb6ca478e51` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:25.344305 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e537e-fd13-41bd-8b53-bef9cf661b4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec5703dfcb5e4ff8826addb6ca478e51)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:25.348318 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e537e-fd13-41bd-8b53-bef9cf661b4b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:25.438120 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:45:25.447977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-104e537e-fd13-41bd-8b53-bef9cf661b4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec5703dfcb5e4ff8826addb6ca478e51', 'request_id': 'req-104e537e-fd13-41bd-8b53-bef9cf661b4b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:25.454674 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-104e537e-fd13-41bd-8b53-bef9cf661b4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec5703dfcb5e4ff8826addb6ca478e51', 'request_id': 'req-104e537e-fd13-41bd-8b53-bef9cf661b4b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:25.509314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-104e537e-fd13-41bd-8b53-bef9cf661b4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec5703dfcb5e4ff8826addb6ca478e51', 'request_id': 'req-104e537e-fd13-41bd-8b53-bef9cf661b4b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:25.533405 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-104e537e-fd13-41bd-8b53-bef9cf661b4b admin admin] The token cache is being invalidate because project ec5703dfcb5e4ff8826addb6ca478e51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:45:25.533405 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-104e537e-fd13-41bd-8b53-bef9cf661b4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:25.533405 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-104e537e-fd13-41bd-8b53-bef9cf661b4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:25.533405 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1322/2615] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:45:25 2022] DELETE /identity/v3/projects/ec5703dfcb5e4ff8826addb6ca478e51 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:45:35.543497 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:45:35.761043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:45:43.179266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32441c-a14e-4beb-8750-3f54dedf5a50 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:43.188120 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32441c-a14e-4beb-8750-3f54dedf5a50 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:43.236626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e55307-1b18-426d-9bd2-e8901f80aa73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:43.237363 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e55307-1b18-426d-9bd2-e8901f80aa73 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:43.237716 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e55307-1b18-426d-9bd2-e8901f80aa73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:43.238096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e55307-1b18-426d-9bd2-e8901f80aa73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:43.239877 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a6e55307-1b18-426d-9bd2-e8901f80aa73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:43.301869 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e55307-1b18-426d-9bd2-e8901f80aa73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:43.303289 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e55307-1b18-426d-9bd2-e8901f80aa73 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:43.313369 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1294/2616] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:45:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:45:46.832602 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e537e-fd13-41bd-8b53-bef9cf661b4b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:46.833271 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-104e537e-fd13-41bd-8b53-bef9cf661b4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:46.847035 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e537e-fd13-41bd-8b53-bef9cf661b4b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:46.851682 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be7d2d-379c-41be-8f58-7451effd3dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:46.852656 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be7d2d-379c-41be-8f58-7451effd3dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:46.852941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be7d2d-379c-41be-8f58-7451effd3dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:46.853253 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be7d2d-379c-41be-8f58-7451effd3dc7 admin admin] PATH_INFO: `/v3/users/0b2efb9857cf495f8b51a96b5f868d44` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:46.854519 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be7d2d-379c-41be-8f58-7451effd3dc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b2efb9857cf495f8b51a96b5f868d44)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:46.856506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be7d2d-379c-41be-8f58-7451effd3dc7 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:47.031707 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-51be7d2d-379c-41be-8f58-7451effd3dc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b2efb9857cf495f8b51a96b5f868d44'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:47.032342 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-51be7d2d-379c-41be-8f58-7451effd3dc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b2efb9857cf495f8b51a96b5f868d44'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:47.039104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-51be7d2d-379c-41be-8f58-7451effd3dc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b2efb9857cf495f8b51a96b5f868d44'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:47.039743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-51be7d2d-379c-41be-8f58-7451effd3dc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b2efb9857cf495f8b51a96b5f868d44'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:47.137594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-51be7d2d-379c-41be-8f58-7451effd3dc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b2efb9857cf495f8b51a96b5f868d44'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:47.156535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1323/2617] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:45:46 2022] DELETE /identity/v3/users/0b2efb9857cf495f8b51a96b5f868d44 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:45:47.419071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e55307-1b18-426d-9bd2-e8901f80aa73 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:47.429368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e55307-1b18-426d-9bd2-e8901f80aa73 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:47.485494 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c893818-35b8-46dd-b726-0723efa9914f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:47.486353 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c893818-35b8-46dd-b726-0723efa9914f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:47.486630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c893818-35b8-46dd-b726-0723efa9914f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:47.486900 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c893818-35b8-46dd-b726-0723efa9914f admin admin] PATH_INFO: `/v3/projects/4f25f022755d465fac40f2681eaef0a5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:47.488065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c893818-35b8-46dd-b726-0723efa9914f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f25f022755d465fac40f2681eaef0a5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:47.489994 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c893818-35b8-46dd-b726-0723efa9914f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:47.617693 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0c893818-35b8-46dd-b726-0723efa9914f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f25f022755d465fac40f2681eaef0a5', 'request_id': 'req-0c893818-35b8-46dd-b726-0723efa9914f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:47.619601 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:45:47.623261 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0c893818-35b8-46dd-b726-0723efa9914f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f25f022755d465fac40f2681eaef0a5', 'request_id': 'req-0c893818-35b8-46dd-b726-0723efa9914f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:47.629942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0c893818-35b8-46dd-b726-0723efa9914f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f25f022755d465fac40f2681eaef0a5', 'request_id': 'req-0c893818-35b8-46dd-b726-0723efa9914f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:47.702418 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0c893818-35b8-46dd-b726-0723efa9914f admin admin] The token cache is being invalidate because project 4f25f022755d465fac40f2681eaef0a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:45:47.702707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0c893818-35b8-46dd-b726-0723efa9914f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:47.703322 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0c893818-35b8-46dd-b726-0723efa9914f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:45:47.704855 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1295/2618] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:45:47 2022] DELETE /identity/v3/projects/4f25f022755d465fac40f2681eaef0a5 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:45:47.709807 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dee744-6340-4452-ac3c-bdeb868e0e0b None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:47.710095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dee744-6340-4452-ac3c-bdeb868e0e0b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:47.710375 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dee744-6340-4452-ac3c-bdeb868e0e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:47.728145 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-04dee744-6340-4452-ac3c-bdeb868e0e0b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:45:47.729380 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-04dee744-6340-4452-ac3c-bdeb868e0e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:47.787934 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1324/2619] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:45:47.796734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1365b-d8f4-47c6-8cbe-ed1affeabb74 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:47.797044 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1365b-d8f4-47c6-8cbe-ed1affeabb74 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:47.797326 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1365b-d8f4-47c6-8cbe-ed1affeabb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:47.815751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f8f1365b-d8f4-47c6-8cbe-ed1affeabb74 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:45:47.816971 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f8f1365b-d8f4-47c6-8cbe-ed1affeabb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:47.825063 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1296/2620] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:45:47.829289 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dee744-6340-4452-ac3c-bdeb868e0e0b None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:47.838945 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dee744-6340-4452-ac3c-bdeb868e0e0b None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:47.843896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a4da5-283f-4898-9829-4188bece234c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:47.844537 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a4da5-283f-4898-9829-4188bece234c admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:47.844816 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a4da5-283f-4898-9829-4188bece234c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:47.845138 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a4da5-283f-4898-9829-4188bece234c admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:47.845858 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a4da5-283f-4898-9829-4188bece234c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:45:47.846160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a4da5-283f-4898-9829-4188bece234c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:47.847745 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a4da5-283f-4898-9829-4188bece234c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:47.861906 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1325/2621] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:45:47.866240 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1365b-d8f4-47c6-8cbe-ed1affeabb74 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:47.876165 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1365b-d8f4-47c6-8cbe-ed1affeabb74 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:47.880800 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f5182-2c20-4512-b6a9-1a14bd7073d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:47.881492 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f5182-2c20-4512-b6a9-1a14bd7073d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:47.881771 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f5182-2c20-4512-b6a9-1a14bd7073d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:47.882048 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f5182-2c20-4512-b6a9-1a14bd7073d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:47.882736 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f5182-2c20-4512-b6a9-1a14bd7073d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:47.884630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f5182-2c20-4512-b6a9-1a14bd7073d0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:48.014246 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1297/2622] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:45:47 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:45:48.018339 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a4da5-283f-4898-9829-4188bece234c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:48.028345 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a4da5-283f-4898-9829-4188bece234c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:48.084004 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e730f9-5e50-4980-bec9-bcfddf511b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:48.084672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e730f9-5e50-4980-bec9-bcfddf511b7f admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:48.084955 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e730f9-5e50-4980-bec9-bcfddf511b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:48.085265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e730f9-5e50-4980-bec9-bcfddf511b7f admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:48.085961 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e730f9-5e50-4980-bec9-bcfddf511b7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:48.087920 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e730f9-5e50-4980-bec9-bcfddf511b7f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:48.095298 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:45:48.172986 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1326/2623] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:45:48 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:45:48.177257 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f5182-2c20-4512-b6a9-1a14bd7073d0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:48.188101 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f5182-2c20-4512-b6a9-1a14bd7073d0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:48.193984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95946f99-ae4d-41ac-92ef-f1cf6bca1af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:48.194684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95946f99-ae4d-41ac-92ef-f1cf6bca1af7 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:48.194966 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95946f99-ae4d-41ac-92ef-f1cf6bca1af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:48.195240 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95946f99-ae4d-41ac-92ef-f1cf6bca1af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:48.195967 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95946f99-ae4d-41ac-92ef-f1cf6bca1af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:48.197620 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95946f99-ae4d-41ac-92ef-f1cf6bca1af7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:48.214770 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1298/2624] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:45:48.219336 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e730f9-5e50-4980-bec9-bcfddf511b7f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:48.230288 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e730f9-5e50-4980-bec9-bcfddf511b7f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:48.236537 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34ecda-5dc1-41c6-a499-1158a7ece9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:48.237244 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34ecda-5dc1-41c6-a499-1158a7ece9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:48.237532 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34ecda-5dc1-41c6-a499-1158a7ece9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:48.237811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34ecda-5dc1-41c6-a499-1158a7ece9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:48.238546 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34ecda-5dc1-41c6-a499-1158a7ece9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:48.240269 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34ecda-5dc1-41c6-a499-1158a7ece9ee admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:48.250589 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1327/2625] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:45:48.254995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95946f99-ae4d-41ac-92ef-f1cf6bca1af7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:48.266758 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95946f99-ae4d-41ac-92ef-f1cf6bca1af7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:48.271971 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba35a1ea-0963-43d9-a922-fea404f1be84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:48.272929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35a1ea-0963-43d9-a922-fea404f1be84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:48.273254 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35a1ea-0963-43d9-a922-fea404f1be84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:48.273526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35a1ea-0963-43d9-a922-fea404f1be84 admin admin] PATH_INFO: `/v3/projects/70af6d2f0ff04bd19ef0e4ba5a0a1ae5/users/eec1fc4f0a87411692b42a6d01017d06/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:48.288898 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba35a1ea-0963-43d9-a922-fea404f1be84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70af6d2f0ff04bd19ef0e4ba5a0a1ae5, user_id=eec1fc4f0a87411692b42a6d01017d06, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:48.291553 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba35a1ea-0963-43d9-a922-fea404f1be84 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:48.365938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1299/2626] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:45:48 2022] PUT /identity/v3/projects/70af6d2f0ff04bd19ef0e4ba5a0a1ae5/users/eec1fc4f0a87411692b42a6d01017d06/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:45:48.375882 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb40d44-ac17-4aa0-8a7f-89ea782bcb7a None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:48.376178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb40d44-ac17-4aa0-8a7f-89ea782bcb7a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:48.376441 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb40d44-ac17-4aa0-8a7f-89ea782bcb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:48.403047 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-efb40d44-ac17-4aa0-8a7f-89ea782bcb7a None None] MFA Rules not processed for user `eec1fc4f0a87411692b42a6d01017d06`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:45:48.404490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-efb40d44-ac17-4aa0-8a7f-89ea782bcb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:48.466083 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1328/2627] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:45:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:45:48.471224 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba35a1ea-0963-43d9-a922-fea404f1be84 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:48.533437 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba35a1ea-0963-43d9-a922-fea404f1be84 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:48.538021 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5d213-da7a-45f3-b9e7-19fa5a6e13b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:48.538673 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5d213-da7a-45f3-b9e7-19fa5a6e13b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:48.538961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5d213-da7a-45f3-b9e7-19fa5a6e13b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:48.539236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5d213-da7a-45f3-b9e7-19fa5a6e13b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:48.539958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5d213-da7a-45f3-b9e7-19fa5a6e13b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:48.541903 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5d213-da7a-45f3-b9e7-19fa5a6e13b1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:48.675931 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1300/2628] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:45:48 2022] POST /identity/v3/projects => generated 379 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:45:48.679954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb40d44-ac17-4aa0-8a7f-89ea782bcb7a None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:48.691004 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb40d44-ac17-4aa0-8a7f-89ea782bcb7a None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:48.746343 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783cb4f-52b0-45f1-95d0-17938bbce126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:48.746998 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783cb4f-52b0-45f1-95d0-17938bbce126 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:48.747279 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783cb4f-52b0-45f1-95d0-17938bbce126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:48.747540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783cb4f-52b0-45f1-95d0-17938bbce126 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:48.748211 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783cb4f-52b0-45f1-95d0-17938bbce126 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:48.750657 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783cb4f-52b0-45f1-95d0-17938bbce126 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:48.836458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1329/2629] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:45:48 2022] POST /identity/v3/users => generated 296 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:45:48.838678 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5d213-da7a-45f3-b9e7-19fa5a6e13b1 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:48.849019 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5d213-da7a-45f3-b9e7-19fa5a6e13b1 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:48.853816 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436645d9-18ed-45ca-a463-79a7820e26b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:48.854468 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436645d9-18ed-45ca-a463-79a7820e26b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:48.854819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436645d9-18ed-45ca-a463-79a7820e26b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:48.855186 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436645d9-18ed-45ca-a463-79a7820e26b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:48.856154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436645d9-18ed-45ca-a463-79a7820e26b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:48.858367 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436645d9-18ed-45ca-a463-79a7820e26b1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:48.870062 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1301/2630] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:45:48.874903 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783cb4f-52b0-45f1-95d0-17938bbce126 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:48.885842 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783cb4f-52b0-45f1-95d0-17938bbce126 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:48.891218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca006b-8ed7-4846-a6ef-97fa72b4b166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:48.892151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca006b-8ed7-4846-a6ef-97fa72b4b166 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:48.892514 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca006b-8ed7-4846-a6ef-97fa72b4b166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:48.892788 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca006b-8ed7-4846-a6ef-97fa72b4b166 admin admin] PATH_INFO: `/v3/projects/8d81760236b841a0bf713394a046fdae/users/964f23360c354acaaab96b34d36306f7/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:48.907751 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca006b-8ed7-4846-a6ef-97fa72b4b166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d81760236b841a0bf713394a046fdae, user_id=964f23360c354acaaab96b34d36306f7, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:48.910477 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca006b-8ed7-4846-a6ef-97fa72b4b166 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:49.011528 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1330/2631] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:45:48 2022] PUT /identity/v3/projects/8d81760236b841a0bf713394a046fdae/users/964f23360c354acaaab96b34d36306f7/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:45:49.015438 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436645d9-18ed-45ca-a463-79a7820e26b1 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:49.025303 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436645d9-18ed-45ca-a463-79a7820e26b1 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:49.073281 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:45:49.082643 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca804196-0f0d-4c72-a767-d99a2a481572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:49.083302 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca804196-0f0d-4c72-a767-d99a2a481572 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:49.092910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca804196-0f0d-4c72-a767-d99a2a481572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:49.092910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca804196-0f0d-4c72-a767-d99a2a481572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:49.092910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca804196-0f0d-4c72-a767-d99a2a481572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:49.092910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca804196-0f0d-4c72-a767-d99a2a481572 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:49.096283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1302/2632] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:45:49.100500 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca006b-8ed7-4846-a6ef-97fa72b4b166 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:49.111701 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca006b-8ed7-4846-a6ef-97fa72b4b166 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:49.116444 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ffa0ec-f936-4cf6-adf1-58c90677bc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:49.117360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffa0ec-f936-4cf6-adf1-58c90677bc33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:49.124291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffa0ec-f936-4cf6-adf1-58c90677bc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:49.124291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffa0ec-f936-4cf6-adf1-58c90677bc33 admin admin] PATH_INFO: `/v3/projects/8d81760236b841a0bf713394a046fdae/users/964f23360c354acaaab96b34d36306f7/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:49.124291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ffa0ec-f936-4cf6-adf1-58c90677bc33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d81760236b841a0bf713394a046fdae, user_id=964f23360c354acaaab96b34d36306f7, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:49.124291 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ffa0ec-f936-4cf6-adf1-58c90677bc33 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:49.132942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1331/2633] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:45:49 2022] PUT /identity/v3/projects/8d81760236b841a0bf713394a046fdae/users/964f23360c354acaaab96b34d36306f7/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:45:49.144684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba84c78-0b6d-4df6-a78c-d65be0fb0ccf None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:49.144972 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba84c78-0b6d-4df6-a78c-d65be0fb0ccf None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:49.145282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba84c78-0b6d-4df6-a78c-d65be0fb0ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:49.172591 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-aba84c78-0b6d-4df6-a78c-d65be0fb0ccf None None] MFA Rules not processed for user `964f23360c354acaaab96b34d36306f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:45:49.173838 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-aba84c78-0b6d-4df6-a78c-d65be0fb0ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:49.243360 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1303/2634] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:45:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:45:49.255324 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ffa0ec-f936-4cf6-adf1-58c90677bc33 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:49.257063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-99ffa0ec-f936-4cf6-adf1-58c90677bc33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:45:49.281062 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:45:49.317643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ffa0ec-f936-4cf6-adf1-58c90677bc33 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:49.321335 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208a3b8-54b8-4405-9e72-b985444be4d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:49.326148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208a3b8-54b8-4405-9e72-b985444be4d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:49.326148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208a3b8-54b8-4405-9e72-b985444be4d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:49.326148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208a3b8-54b8-4405-9e72-b985444be4d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:49.332238 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208a3b8-54b8-4405-9e72-b985444be4d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:49.333707 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208a3b8-54b8-4405-9e72-b985444be4d6 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:49.344034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1332/2635] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:45:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:45:49.550283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba84c78-0b6d-4df6-a78c-d65be0fb0ccf None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:45:49.559780 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba84c78-0b6d-4df6-a78c-d65be0fb0ccf None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:45:49.563820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9470bda-63df-4044-bd0a-69910d3beefb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:45:49.565411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9470bda-63df-4044-bd0a-69910d3beefb service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:45:49.565411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9470bda-63df-4044-bd0a-69910d3beefb service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:45:49.565411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9470bda-63df-4044-bd0a-69910d3beefb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:45:49.621199 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9470bda-63df-4044-bd0a-69910d3beefb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:45:49.622971 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9470bda-63df-4044-bd0a-69910d3beefb service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:45:49.632316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1304/2636] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:45:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:45:59.345170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=17 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:45:59.628279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=68 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:00.979112 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208a3b8-54b8-4405-9e72-b985444be4d6 service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:01.042660 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208a3b8-54b8-4405-9e72-b985444be4d6 service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:01.047545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b62bc3-2d48-4604-89f1-2c3ab4c577cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:01.049265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b62bc3-2d48-4604-89f1-2c3ab4c577cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:01.049265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b62bc3-2d48-4604-89f1-2c3ab4c577cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:01.049265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b62bc3-2d48-4604-89f1-2c3ab4c577cf admin admin] PATH_INFO: `/v3/users/eec1fc4f0a87411692b42a6d01017d06` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:01.050209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b62bc3-2d48-4604-89f1-2c3ab4c577cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=eec1fc4f0a87411692b42a6d01017d06)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:01.052117 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b62bc3-2d48-4604-89f1-2c3ab4c577cf admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:01.247811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-96b62bc3-2d48-4604-89f1-2c3ab4c577cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eec1fc4f0a87411692b42a6d01017d06'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:01.248379 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-96b62bc3-2d48-4604-89f1-2c3ab4c577cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eec1fc4f0a87411692b42a6d01017d06'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:01.257220 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-96b62bc3-2d48-4604-89f1-2c3ab4c577cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eec1fc4f0a87411692b42a6d01017d06'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:01.257870 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-96b62bc3-2d48-4604-89f1-2c3ab4c577cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eec1fc4f0a87411692b42a6d01017d06'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:01.342594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-96b62bc3-2d48-4604-89f1-2c3ab4c577cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eec1fc4f0a87411692b42a6d01017d06'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:01.360051 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1333/2637] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:46:00 2022] DELETE /identity/v3/users/eec1fc4f0a87411692b42a6d01017d06 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:01.364672 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689057aa-4b62-4bfa-9099-8d4abcccbfb0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:01.365177 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689057aa-4b62-4bfa-9099-8d4abcccbfb0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:01.365293 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689057aa-4b62-4bfa-9099-8d4abcccbfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:01.383097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-689057aa-4b62-4bfa-9099-8d4abcccbfb0 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:01.384296 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-689057aa-4b62-4bfa-9099-8d4abcccbfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:01.444161 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1305/2638] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:46:01.454723 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b62bc3-2d48-4604-89f1-2c3ab4c577cf admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:01.456191 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-96b62bc3-2d48-4604-89f1-2c3ab4c577cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:01.512813 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b62bc3-2d48-4604-89f1-2c3ab4c577cf admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:01.516350 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971a5f9-1132-4535-89d2-d583473342b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:01.517109 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971a5f9-1132-4535-89d2-d583473342b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:01.517396 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971a5f9-1132-4535-89d2-d583473342b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:01.517671 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971a5f9-1132-4535-89d2-d583473342b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:01.526686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971a5f9-1132-4535-89d2-d583473342b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:01.528154 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971a5f9-1132-4535-89d2-d583473342b1 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:01.542701 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1334/2639] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:46:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:46:01.933720 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689057aa-4b62-4bfa-9099-8d4abcccbfb0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:01.935251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-689057aa-4b62-4bfa-9099-8d4abcccbfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:01.947776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689057aa-4b62-4bfa-9099-8d4abcccbfb0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:01.953753 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d70d5-3a42-446f-a964-c978ceda5879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:01.954633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d70d5-3a42-446f-a964-c978ceda5879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:01.954915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d70d5-3a42-446f-a964-c978ceda5879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:01.955190 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d70d5-3a42-446f-a964-c978ceda5879 admin admin] PATH_INFO: `/v3/projects/70af6d2f0ff04bd19ef0e4ba5a0a1ae5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:01.956965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920d70d5-3a42-446f-a964-c978ceda5879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70af6d2f0ff04bd19ef0e4ba5a0a1ae5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:01.959138 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920d70d5-3a42-446f-a964-c978ceda5879 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:02.074236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-920d70d5-3a42-446f-a964-c978ceda5879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70af6d2f0ff04bd19ef0e4ba5a0a1ae5', 'request_id': 'req-920d70d5-3a42-446f-a964-c978ceda5879'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:02.079592 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-920d70d5-3a42-446f-a964-c978ceda5879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70af6d2f0ff04bd19ef0e4ba5a0a1ae5', 'request_id': 'req-920d70d5-3a42-446f-a964-c978ceda5879'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:02.085951 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-920d70d5-3a42-446f-a964-c978ceda5879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70af6d2f0ff04bd19ef0e4ba5a0a1ae5', 'request_id': 'req-920d70d5-3a42-446f-a964-c978ceda5879'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:02.171024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-920d70d5-3a42-446f-a964-c978ceda5879 admin admin] The token cache is being invalidate because project 70af6d2f0ff04bd19ef0e4ba5a0a1ae5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:46:02.171309 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-920d70d5-3a42-446f-a964-c978ceda5879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:02.172000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-920d70d5-3a42-446f-a964-c978ceda5879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:02.174305 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1306/2640] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:46:01 2022] DELETE /identity/v3/projects/70af6d2f0ff04bd19ef0e4ba5a0a1ae5 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:02.178314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971a5f9-1132-4535-89d2-d583473342b1 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:02.188382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971a5f9-1132-4535-89d2-d583473342b1 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:02.207080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:02.247015 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b3fe22-1df4-4898-9f11-e0e5133fe678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:02.247873 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3fe22-1df4-4898-9f11-e0e5133fe678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:02.248146 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3fe22-1df4-4898-9f11-e0e5133fe678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:02.248414 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3fe22-1df4-4898-9f11-e0e5133fe678 admin admin] PATH_INFO: `/v3/users/964f23360c354acaaab96b34d36306f7` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:02.249615 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b3fe22-1df4-4898-9f11-e0e5133fe678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=964f23360c354acaaab96b34d36306f7)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:02.251544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b3fe22-1df4-4898-9f11-e0e5133fe678 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:02.424565 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-37b3fe22-1df4-4898-9f11-e0e5133fe678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '964f23360c354acaaab96b34d36306f7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:02.425766 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-37b3fe22-1df4-4898-9f11-e0e5133fe678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '964f23360c354acaaab96b34d36306f7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:02.432504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-37b3fe22-1df4-4898-9f11-e0e5133fe678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '964f23360c354acaaab96b34d36306f7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:02.433154 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-37b3fe22-1df4-4898-9f11-e0e5133fe678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '964f23360c354acaaab96b34d36306f7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:02.496790 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-37b3fe22-1df4-4898-9f11-e0e5133fe678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '964f23360c354acaaab96b34d36306f7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:02.514313 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1335/2641] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:46:02 2022] DELETE /identity/v3/users/964f23360c354acaaab96b34d36306f7 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:46:02.922329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d70d5-3a42-446f-a964-c978ceda5879 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:02.924384 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-920d70d5-3a42-446f-a964-c978ceda5879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:02.945431 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d70d5-3a42-446f-a964-c978ceda5879 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:03.016821 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32714ae8-d5fe-4f97-90f0-e68616448835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:03.018158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32714ae8-d5fe-4f97-90f0-e68616448835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:03.018580 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32714ae8-d5fe-4f97-90f0-e68616448835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:03.019009 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32714ae8-d5fe-4f97-90f0-e68616448835 admin admin] PATH_INFO: `/v3/projects/8d81760236b841a0bf713394a046fdae` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:03.020820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32714ae8-d5fe-4f97-90f0-e68616448835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d81760236b841a0bf713394a046fdae)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:03.023985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32714ae8-d5fe-4f97-90f0-e68616448835 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:03.132864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:03.148579 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-32714ae8-d5fe-4f97-90f0-e68616448835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d81760236b841a0bf713394a046fdae', 'request_id': 'req-32714ae8-d5fe-4f97-90f0-e68616448835'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:03.154150 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-32714ae8-d5fe-4f97-90f0-e68616448835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d81760236b841a0bf713394a046fdae', 'request_id': 'req-32714ae8-d5fe-4f97-90f0-e68616448835'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:03.160231 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-32714ae8-d5fe-4f97-90f0-e68616448835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d81760236b841a0bf713394a046fdae', 'request_id': 'req-32714ae8-d5fe-4f97-90f0-e68616448835'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:03.240392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-32714ae8-d5fe-4f97-90f0-e68616448835 admin admin] The token cache is being invalidate because project 8d81760236b841a0bf713394a046fdae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:46:03.240713 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-32714ae8-d5fe-4f97-90f0-e68616448835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:03.241343 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-32714ae8-d5fe-4f97-90f0-e68616448835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:03.242883 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1307/2642] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:46:02 2022] DELETE /identity/v3/projects/8d81760236b841a0bf713394a046fdae => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:46:03.248121 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b9443-fc15-4749-be92-47d6497b9a2b None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:03.248903 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b9443-fc15-4749-be92-47d6497b9a2b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:03.249104 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b9443-fc15-4749-be92-47d6497b9a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:03.268803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-442b9443-fc15-4749-be92-47d6497b9a2b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:03.269975 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-442b9443-fc15-4749-be92-47d6497b9a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:03.331176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1336/2643] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:46:03.339428 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8e542-9c66-4e61-ab59-a1d55a618df0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:03.339713 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8e542-9c66-4e61-ab59-a1d55a618df0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:03.339994 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8e542-9c66-4e61-ab59-a1d55a618df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:03.358674 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-65b8e542-9c66-4e61-ab59-a1d55a618df0 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:03.359895 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-65b8e542-9c66-4e61-ab59-a1d55a618df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:03.369135 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1308/2644] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:46:03.373438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b9443-fc15-4749-be92-47d6497b9a2b None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:03.383392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b9443-fc15-4749-be92-47d6497b9a2b None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:03.388103 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91435fe-a92f-4843-ae06-41d98d8a48dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:03.388754 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91435fe-a92f-4843-ae06-41d98d8a48dd admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:03.389072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91435fe-a92f-4843-ae06-41d98d8a48dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:03.389351 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91435fe-a92f-4843-ae06-41d98d8a48dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:03.390083 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91435fe-a92f-4843-ae06-41d98d8a48dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:46:03.390384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91435fe-a92f-4843-ae06-41d98d8a48dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:03.392144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91435fe-a92f-4843-ae06-41d98d8a48dd admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:03.404390 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1337/2645] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:46:03.409644 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8e542-9c66-4e61-ab59-a1d55a618df0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:03.419077 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8e542-9c66-4e61-ab59-a1d55a618df0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:03.424403 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b9536-73d5-46f4-a360-cbc92b75ea02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:03.425093 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b9536-73d5-46f4-a360-cbc92b75ea02 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:03.425382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b9536-73d5-46f4-a360-cbc92b75ea02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:03.425664 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b9536-73d5-46f4-a360-cbc92b75ea02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:03.426348 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b9536-73d5-46f4-a360-cbc92b75ea02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:03.428268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b9536-73d5-46f4-a360-cbc92b75ea02 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:03.539993 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1309/2646] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:46:03 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:46:03.544369 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91435fe-a92f-4843-ae06-41d98d8a48dd admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:03.555182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91435fe-a92f-4843-ae06-41d98d8a48dd admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:03.581332 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:03.613032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fef27-c3ba-47bb-9f66-5f882edd5036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:03.613709 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fef27-c3ba-47bb-9f66-5f882edd5036 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:03.613993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fef27-c3ba-47bb-9f66-5f882edd5036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:03.614284 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fef27-c3ba-47bb-9f66-5f882edd5036 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:03.614964 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fef27-c3ba-47bb-9f66-5f882edd5036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:03.617109 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fef27-c3ba-47bb-9f66-5f882edd5036 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:03.710370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1338/2647] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:46:03 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:46:03.715127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b9536-73d5-46f4-a360-cbc92b75ea02 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:03.726504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b9536-73d5-46f4-a360-cbc92b75ea02 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:03.731306 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d09413-fd22-4179-8bf4-06a0665680e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:03.731965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d09413-fd22-4179-8bf4-06a0665680e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:03.732247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d09413-fd22-4179-8bf4-06a0665680e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:03.732523 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d09413-fd22-4179-8bf4-06a0665680e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:03.733304 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d09413-fd22-4179-8bf4-06a0665680e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:03.735007 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d09413-fd22-4179-8bf4-06a0665680e5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:03.745579 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1310/2648] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:46:03.749773 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fef27-c3ba-47bb-9f66-5f882edd5036 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:03.759954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fef27-c3ba-47bb-9f66-5f882edd5036 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:03.764594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacbae38-f701-482a-8cbb-6065c5efcba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:03.765280 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbae38-f701-482a-8cbb-6065c5efcba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:03.765565 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbae38-f701-482a-8cbb-6065c5efcba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:03.765839 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbae38-f701-482a-8cbb-6065c5efcba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:03.767062 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacbae38-f701-482a-8cbb-6065c5efcba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:03.768664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacbae38-f701-482a-8cbb-6065c5efcba8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:03.778819 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1339/2649] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:46:03.783676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d09413-fd22-4179-8bf4-06a0665680e5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:03.793996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d09413-fd22-4179-8bf4-06a0665680e5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:03.798670 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af100d19-396f-4efd-80f7-d4efd9149caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:03.799555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af100d19-396f-4efd-80f7-d4efd9149caf admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:03.799841 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af100d19-396f-4efd-80f7-d4efd9149caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:03.800140 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af100d19-396f-4efd-80f7-d4efd9149caf admin admin] PATH_INFO: `/v3/projects/d65bf948196e4cc185e77647207f6276/users/78404f359801498ab5432a7e07f184e5/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:03.814847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af100d19-396f-4efd-80f7-d4efd9149caf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d65bf948196e4cc185e77647207f6276, user_id=78404f359801498ab5432a7e07f184e5, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:03.817549 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af100d19-396f-4efd-80f7-d4efd9149caf admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:03.880565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1311/2650] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:46:03 2022] PUT /identity/v3/projects/d65bf948196e4cc185e77647207f6276/users/78404f359801498ab5432a7e07f184e5/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:46:03.892961 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1c529-e05f-416a-8c17-f50fb1a6d255 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:03.893375 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1c529-e05f-416a-8c17-f50fb1a6d255 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:03.893702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1c529-e05f-416a-8c17-f50fb1a6d255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:03.921406 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-62e1c529-e05f-416a-8c17-f50fb1a6d255 None None] MFA Rules not processed for user `78404f359801498ab5432a7e07f184e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:03.922620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-62e1c529-e05f-416a-8c17-f50fb1a6d255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:03.987041 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1340/2651] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:46:03.991700 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af100d19-396f-4efd-80f7-d4efd9149caf admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:04.053515 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af100d19-396f-4efd-80f7-d4efd9149caf admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:04.058146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafdcd2d-976a-4ed4-aa42-6c9cfeae929f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:04.059053 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafdcd2d-976a-4ed4-aa42-6c9cfeae929f admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:04.059343 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafdcd2d-976a-4ed4-aa42-6c9cfeae929f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:04.059618 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafdcd2d-976a-4ed4-aa42-6c9cfeae929f admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:04.060311 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafdcd2d-976a-4ed4-aa42-6c9cfeae929f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:04.062233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafdcd2d-976a-4ed4-aa42-6c9cfeae929f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:04.145462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1312/2652] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:46:03 2022] POST /identity/v3/projects => generated 365 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:46:04.149945 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1c529-e05f-416a-8c17-f50fb1a6d255 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:04.162077 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1c529-e05f-416a-8c17-f50fb1a6d255 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:04.226747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600b861-aa21-484c-9dad-3cbd68c21650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:04.227409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600b861-aa21-484c-9dad-3cbd68c21650 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:04.227691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600b861-aa21-484c-9dad-3cbd68c21650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:04.227962 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600b861-aa21-484c-9dad-3cbd68c21650 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:04.228645 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a600b861-aa21-484c-9dad-3cbd68c21650 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:04.230688 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a600b861-aa21-484c-9dad-3cbd68c21650 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:04.332799 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1341/2653] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:46:04 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:46:04.337016 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafdcd2d-976a-4ed4-aa42-6c9cfeae929f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:04.347578 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafdcd2d-976a-4ed4-aa42-6c9cfeae929f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:04.352090 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208627bc-4950-49b3-8df8-d68b412b0c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:04.352754 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208627bc-4950-49b3-8df8-d68b412b0c54 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:04.353068 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208627bc-4950-49b3-8df8-d68b412b0c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:04.353358 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208627bc-4950-49b3-8df8-d68b412b0c54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:04.354082 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208627bc-4950-49b3-8df8-d68b412b0c54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:04.355731 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208627bc-4950-49b3-8df8-d68b412b0c54 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:04.367396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1313/2654] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:46:04.371630 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600b861-aa21-484c-9dad-3cbd68c21650 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:04.382450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600b861-aa21-484c-9dad-3cbd68c21650 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:04.387227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd0428-4791-48e2-8d87-88ae341c8b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:04.388110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd0428-4791-48e2-8d87-88ae341c8b60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:04.388392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd0428-4791-48e2-8d87-88ae341c8b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:04.388665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd0428-4791-48e2-8d87-88ae341c8b60 admin admin] PATH_INFO: `/v3/projects/010bf491165b49d58f4f0ed2fc50d735/users/1a860266dd3147c68c56847ee55a19c7/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:04.405439 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd0428-4791-48e2-8d87-88ae341c8b60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=010bf491165b49d58f4f0ed2fc50d735, user_id=1a860266dd3147c68c56847ee55a19c7, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:04.408105 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd0428-4791-48e2-8d87-88ae341c8b60 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:04.474339 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1342/2655] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:46:04 2022] PUT /identity/v3/projects/010bf491165b49d58f4f0ed2fc50d735/users/1a860266dd3147c68c56847ee55a19c7/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:04.478384 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208627bc-4950-49b3-8df8-d68b412b0c54 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:04.489360 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208627bc-4950-49b3-8df8-d68b412b0c54 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:04.509882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:04.546702 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74319a-6796-4e54-b440-caf5ff5e34ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:04.547372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74319a-6796-4e54-b440-caf5ff5e34ae admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:04.547652 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74319a-6796-4e54-b440-caf5ff5e34ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:04.548433 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74319a-6796-4e54-b440-caf5ff5e34ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:04.549232 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74319a-6796-4e54-b440-caf5ff5e34ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:04.550931 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74319a-6796-4e54-b440-caf5ff5e34ae admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:04.560836 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1314/2656] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:46:04.566129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd0428-4791-48e2-8d87-88ae341c8b60 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:04.571721 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:04.577230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd0428-4791-48e2-8d87-88ae341c8b60 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:04.581944 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1f203-4534-40cc-b13b-21db21b87529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:04.582935 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1f203-4534-40cc-b13b-21db21b87529 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:04.583170 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1f203-4534-40cc-b13b-21db21b87529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:04.583450 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1f203-4534-40cc-b13b-21db21b87529 admin admin] PATH_INFO: `/v3/projects/010bf491165b49d58f4f0ed2fc50d735/users/1a860266dd3147c68c56847ee55a19c7/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:04.585683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1f203-4534-40cc-b13b-21db21b87529 admin admin] RBAC: Authorizing `identity:create_grant(project_id=010bf491165b49d58f4f0ed2fc50d735, user_id=1a860266dd3147c68c56847ee55a19c7, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:04.588400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1f203-4534-40cc-b13b-21db21b87529 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:04.599095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1343/2657] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:46:04 2022] PUT /identity/v3/projects/010bf491165b49d58f4f0ed2fc50d735/users/1a860266dd3147c68c56847ee55a19c7/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:04.609687 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f2a84-73e5-4b5a-83de-ea50d4d9332f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:04.610028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f2a84-73e5-4b5a-83de-ea50d4d9332f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:04.610986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f2a84-73e5-4b5a-83de-ea50d4d9332f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:04.638030 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b59f2a84-73e5-4b5a-83de-ea50d4d9332f None None] MFA Rules not processed for user `1a860266dd3147c68c56847ee55a19c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:04.639249 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b59f2a84-73e5-4b5a-83de-ea50d4d9332f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:04.706288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1315/2658] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:04 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:46:04.717953 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1f203-4534-40cc-b13b-21db21b87529 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:04.719860 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a9d1f203-4534-40cc-b13b-21db21b87529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:04.777068 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1f203-4534-40cc-b13b-21db21b87529 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:04.781066 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa9711-bab5-4548-b157-021ab3306015 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:04.781709 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa9711-bab5-4548-b157-021ab3306015 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:04.781992 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa9711-bab5-4548-b157-021ab3306015 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:04.782381 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa9711-bab5-4548-b157-021ab3306015 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:04.791408 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa9711-bab5-4548-b157-021ab3306015 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:04.792883 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa9711-bab5-4548-b157-021ab3306015 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:04.803201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1344/2659] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:46:04 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:46:05.644651 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f2a84-73e5-4b5a-83de-ea50d4d9332f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:05.708223 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f2a84-73e5-4b5a-83de-ea50d4d9332f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:05.712774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36722fc4-d87f-4202-9e99-c483828b85f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:05.713682 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36722fc4-d87f-4202-9e99-c483828b85f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:05.713970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36722fc4-d87f-4202-9e99-c483828b85f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:05.714248 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36722fc4-d87f-4202-9e99-c483828b85f9 admin admin] PATH_INFO: `/v3/users/78404f359801498ab5432a7e07f184e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:05.715442 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36722fc4-d87f-4202-9e99-c483828b85f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78404f359801498ab5432a7e07f184e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:05.717406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36722fc4-d87f-4202-9e99-c483828b85f9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:05.934529 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36722fc4-d87f-4202-9e99-c483828b85f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78404f359801498ab5432a7e07f184e5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:05.935171 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36722fc4-d87f-4202-9e99-c483828b85f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78404f359801498ab5432a7e07f184e5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:05.936361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:05.942423 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36722fc4-d87f-4202-9e99-c483828b85f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78404f359801498ab5432a7e07f184e5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:05.943047 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36722fc4-d87f-4202-9e99-c483828b85f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78404f359801498ab5432a7e07f184e5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:05.955048 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36722fc4-d87f-4202-9e99-c483828b85f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78404f359801498ab5432a7e07f184e5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:06.052048 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1316/2660] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:46:05 2022] DELETE /identity/v3/users/78404f359801498ab5432a7e07f184e5 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:06.057108 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb2e47-d7ce-4019-ae80-7e5dfc85004d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:06.057605 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb2e47-d7ce-4019-ae80-7e5dfc85004d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:06.057859 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb2e47-d7ce-4019-ae80-7e5dfc85004d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:06.077090 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-4fbb2e47-d7ce-4019-ae80-7e5dfc85004d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:06.078298 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4fbb2e47-d7ce-4019-ae80-7e5dfc85004d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:06.142059 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1345/2661] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:46:06.151978 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36722fc4-d87f-4202-9e99-c483828b85f9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:06.153684 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-36722fc4-d87f-4202-9e99-c483828b85f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:06.217401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36722fc4-d87f-4202-9e99-c483828b85f9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:06.221641 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf893698-f125-4264-832a-ae42f8270095 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:06.222316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf893698-f125-4264-832a-ae42f8270095 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:06.222595 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf893698-f125-4264-832a-ae42f8270095 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:06.222872 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf893698-f125-4264-832a-ae42f8270095 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:06.232143 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf893698-f125-4264-832a-ae42f8270095 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:06.233743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf893698-f125-4264-832a-ae42f8270095 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:06.243339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1317/2662] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:46:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:46:06.648428 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb2e47-d7ce-4019-ae80-7e5dfc85004d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:06.649893 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4fbb2e47-d7ce-4019-ae80-7e5dfc85004d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:06.662560 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb2e47-d7ce-4019-ae80-7e5dfc85004d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:06.667541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a6820-6cc0-4c8e-92c5-73a944b615b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:06.668415 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a6820-6cc0-4c8e-92c5-73a944b615b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:06.668704 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a6820-6cc0-4c8e-92c5-73a944b615b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:06.669176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a6820-6cc0-4c8e-92c5-73a944b615b9 admin admin] PATH_INFO: `/v3/projects/d65bf948196e4cc185e77647207f6276` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:06.670388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a6820-6cc0-4c8e-92c5-73a944b615b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d65bf948196e4cc185e77647207f6276)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:06.672326 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a6820-6cc0-4c8e-92c5-73a944b615b9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:06.766779 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-771a6820-6cc0-4c8e-92c5-73a944b615b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd65bf948196e4cc185e77647207f6276', 'request_id': 'req-771a6820-6cc0-4c8e-92c5-73a944b615b9'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:06.773396 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-771a6820-6cc0-4c8e-92c5-73a944b615b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd65bf948196e4cc185e77647207f6276', 'request_id': 'req-771a6820-6cc0-4c8e-92c5-73a944b615b9'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:06.823012 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-771a6820-6cc0-4c8e-92c5-73a944b615b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd65bf948196e4cc185e77647207f6276', 'request_id': 'req-771a6820-6cc0-4c8e-92c5-73a944b615b9'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:06.835187 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-771a6820-6cc0-4c8e-92c5-73a944b615b9 admin admin] The token cache is being invalidate because project d65bf948196e4cc185e77647207f6276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:46:06.835506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-771a6820-6cc0-4c8e-92c5-73a944b615b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:06.836136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-771a6820-6cc0-4c8e-92c5-73a944b615b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:06.837962 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1346/2663] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:46:06 2022] DELETE /identity/v3/projects/d65bf948196e4cc185e77647207f6276 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:06.842077 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf893698-f125-4264-832a-ae42f8270095 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:06.851805 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf893698-f125-4264-832a-ae42f8270095 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:06.910492 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa7b5d-a0dd-4076-b050-cacfd5909c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:06.911279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa7b5d-a0dd-4076-b050-cacfd5909c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:06.911565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa7b5d-a0dd-4076-b050-cacfd5909c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:06.911842 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa7b5d-a0dd-4076-b050-cacfd5909c73 admin admin] PATH_INFO: `/v3/users/1a860266dd3147c68c56847ee55a19c7` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:06.913059 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa7b5d-a0dd-4076-b050-cacfd5909c73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a860266dd3147c68c56847ee55a19c7)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:06.915002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa7b5d-a0dd-4076-b050-cacfd5909c73 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:07.083236 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1bfa7b5d-a0dd-4076-b050-cacfd5909c73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a860266dd3147c68c56847ee55a19c7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:07.083894 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1bfa7b5d-a0dd-4076-b050-cacfd5909c73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a860266dd3147c68c56847ee55a19c7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:07.091392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1bfa7b5d-a0dd-4076-b050-cacfd5909c73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a860266dd3147c68c56847ee55a19c7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:07.092081 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1bfa7b5d-a0dd-4076-b050-cacfd5909c73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a860266dd3147c68c56847ee55a19c7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:07.103300 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:07.105648 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1bfa7b5d-a0dd-4076-b050-cacfd5909c73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a860266dd3147c68c56847ee55a19c7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:07.154572 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1318/2664] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:46:06 2022] DELETE /identity/v3/users/1a860266dd3147c68c56847ee55a19c7 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:07.560095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a6820-6cc0-4c8e-92c5-73a944b615b9 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:07.560095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-771a6820-6cc0-4c8e-92c5-73a944b615b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:07.575777 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a6820-6cc0-4c8e-92c5-73a944b615b9 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:07.590971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:07.640102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679a321-39c1-4965-b0f9-2d2ddd477683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:07.640971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679a321-39c1-4965-b0f9-2d2ddd477683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:07.641665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679a321-39c1-4965-b0f9-2d2ddd477683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:07.641790 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679a321-39c1-4965-b0f9-2d2ddd477683 admin admin] PATH_INFO: `/v3/projects/010bf491165b49d58f4f0ed2fc50d735` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:07.643002 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679a321-39c1-4965-b0f9-2d2ddd477683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=010bf491165b49d58f4f0ed2fc50d735)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:07.644948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679a321-39c1-4965-b0f9-2d2ddd477683 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:07.726601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7679a321-39c1-4965-b0f9-2d2ddd477683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '010bf491165b49d58f4f0ed2fc50d735', 'request_id': 'req-7679a321-39c1-4965-b0f9-2d2ddd477683'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:07.732989 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7679a321-39c1-4965-b0f9-2d2ddd477683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '010bf491165b49d58f4f0ed2fc50d735', 'request_id': 'req-7679a321-39c1-4965-b0f9-2d2ddd477683'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:07.771123 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7679a321-39c1-4965-b0f9-2d2ddd477683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '010bf491165b49d58f4f0ed2fc50d735', 'request_id': 'req-7679a321-39c1-4965-b0f9-2d2ddd477683'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:07.782664 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7679a321-39c1-4965-b0f9-2d2ddd477683 admin admin] The token cache is being invalidate because project 010bf491165b49d58f4f0ed2fc50d735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:46:07.782945 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7679a321-39c1-4965-b0f9-2d2ddd477683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:07.783624 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7679a321-39c1-4965-b0f9-2d2ddd477683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:07.785264 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1347/2665] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:46:07 2022] DELETE /identity/v3/projects/010bf491165b49d58f4f0ed2fc50d735 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:46:07.790447 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e785497-b27c-407d-9ce1-0d0a0ae826ec None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:07.790752 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e785497-b27c-407d-9ce1-0d0a0ae826ec None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:07.791037 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e785497-b27c-407d-9ce1-0d0a0ae826ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:07.810468 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7e785497-b27c-407d-9ce1-0d0a0ae826ec None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:07.811701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7e785497-b27c-407d-9ce1-0d0a0ae826ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:07.872128 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1319/2666] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:46:07.880152 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53205fc1-6c39-4c85-8096-4b27d6555de6 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:07.880445 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53205fc1-6c39-4c85-8096-4b27d6555de6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:07.880727 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53205fc1-6c39-4c85-8096-4b27d6555de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:07.899782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-53205fc1-6c39-4c85-8096-4b27d6555de6 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:07.901039 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-53205fc1-6c39-4c85-8096-4b27d6555de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:07.910086 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1348/2667] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:46:07.914478 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e785497-b27c-407d-9ce1-0d0a0ae826ec None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:07.924851 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e785497-b27c-407d-9ce1-0d0a0ae826ec None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:07.929498 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511da2be-2466-43a2-816e-e81d2a2e6ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:07.930152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511da2be-2466-43a2-816e-e81d2a2e6ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:07.930435 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511da2be-2466-43a2-816e-e81d2a2e6ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:07.930713 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511da2be-2466-43a2-816e-e81d2a2e6ee2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:07.931442 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511da2be-2466-43a2-816e-e81d2a2e6ee2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:46:07.931751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511da2be-2466-43a2-816e-e81d2a2e6ee2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:07.933391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511da2be-2466-43a2-816e-e81d2a2e6ee2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:07.945476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1320/2668] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:46:07.949759 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53205fc1-6c39-4c85-8096-4b27d6555de6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:07.959454 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53205fc1-6c39-4c85-8096-4b27d6555de6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:07.964178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547fb24f-4c40-4ecd-b860-6041b4fa42b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:07.964830 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fb24f-4c40-4ecd-b860-6041b4fa42b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:07.965151 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fb24f-4c40-4ecd-b860-6041b4fa42b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:07.965425 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fb24f-4c40-4ecd-b860-6041b4fa42b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:07.966117 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547fb24f-4c40-4ecd-b860-6041b4fa42b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:07.968036 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547fb24f-4c40-4ecd-b860-6041b4fa42b8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:08.041919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1349/2669] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:46:07 2022] POST /identity/v3/projects => generated 369 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:46:08.046562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511da2be-2466-43a2-816e-e81d2a2e6ee2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:08.057509 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511da2be-2466-43a2-816e-e81d2a2e6ee2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:08.116059 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea199f2-3e94-49bc-aec7-f3765c140b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:08.116774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea199f2-3e94-49bc-aec7-f3765c140b9b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:08.117094 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea199f2-3e94-49bc-aec7-f3765c140b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:08.117378 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea199f2-3e94-49bc-aec7-f3765c140b9b admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:08.118071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea199f2-3e94-49bc-aec7-f3765c140b9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:08.119998 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea199f2-3e94-49bc-aec7-f3765c140b9b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:08.200313 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1321/2670] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:46:08 2022] POST /identity/v3/users => generated 285 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:46:08.205305 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547fb24f-4c40-4ecd-b860-6041b4fa42b8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:08.215998 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547fb24f-4c40-4ecd-b860-6041b4fa42b8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:08.221351 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae5ca5-952c-4a78-8690-5c7864a6a6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:08.222007 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae5ca5-952c-4a78-8690-5c7864a6a6e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:08.222284 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae5ca5-952c-4a78-8690-5c7864a6a6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:08.222556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae5ca5-952c-4a78-8690-5c7864a6a6e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:08.223281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae5ca5-952c-4a78-8690-5c7864a6a6e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:08.224872 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae5ca5-952c-4a78-8690-5c7864a6a6e1 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:08.235665 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1350/2671] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:46:08.240059 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea199f2-3e94-49bc-aec7-f3765c140b9b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:08.250165 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea199f2-3e94-49bc-aec7-f3765c140b9b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:08.254751 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b403117-6577-4bed-bb75-ab4ca62f1c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:08.255408 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b403117-6577-4bed-bb75-ab4ca62f1c0d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:08.255686 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b403117-6577-4bed-bb75-ab4ca62f1c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:08.255954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b403117-6577-4bed-bb75-ab4ca62f1c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:08.256666 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b403117-6577-4bed-bb75-ab4ca62f1c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:08.258318 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b403117-6577-4bed-bb75-ab4ca62f1c0d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:08.269166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1322/2672] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:46:08.273311 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae5ca5-952c-4a78-8690-5c7864a6a6e1 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:08.284006 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae5ca5-952c-4a78-8690-5c7864a6a6e1 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:08.288564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531233ab-7a6c-4dd5-ad57-a96893beca48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:08.289477 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531233ab-7a6c-4dd5-ad57-a96893beca48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:08.289754 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531233ab-7a6c-4dd5-ad57-a96893beca48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:08.290028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531233ab-7a6c-4dd5-ad57-a96893beca48 admin admin] PATH_INFO: `/v3/projects/77a71fdc2a8341859a1d45f3f53f3301/users/c1f38e2933c449189a221be793e55aee/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:08.304609 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531233ab-7a6c-4dd5-ad57-a96893beca48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77a71fdc2a8341859a1d45f3f53f3301, user_id=c1f38e2933c449189a221be793e55aee, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:08.307209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531233ab-7a6c-4dd5-ad57-a96893beca48 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:08.406671 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1351/2673] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:46:08 2022] PUT /identity/v3/projects/77a71fdc2a8341859a1d45f3f53f3301/users/c1f38e2933c449189a221be793e55aee/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:46:08.418765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3cc48-c59f-4fae-8e65-e9dfff7d7ecd None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:08.419108 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3cc48-c59f-4fae-8e65-e9dfff7d7ecd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:08.419390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3cc48-c59f-4fae-8e65-e9dfff7d7ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:08.445970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-55b3cc48-c59f-4fae-8e65-e9dfff7d7ecd None None] MFA Rules not processed for user `c1f38e2933c449189a221be793e55aee`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:08.447182 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-55b3cc48-c59f-4fae-8e65-e9dfff7d7ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:08.490013 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:08.512874 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1323/2674] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:08 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:46:08.518116 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531233ab-7a6c-4dd5-ad57-a96893beca48 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:08.565337 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:08.583730 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531233ab-7a6c-4dd5-ad57-a96893beca48 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:08.588776 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96258979-57ed-499f-9c9c-c185ca6e9c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:08.589702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96258979-57ed-499f-9c9c-c185ca6e9c53 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:08.589990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96258979-57ed-499f-9c9c-c185ca6e9c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:08.590269 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96258979-57ed-499f-9c9c-c185ca6e9c53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:08.590971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96258979-57ed-499f-9c9c-c185ca6e9c53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:08.592956 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96258979-57ed-499f-9c9c-c185ca6e9c53 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:08.652952 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1352/2675] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:46:08 2022] POST /identity/v3/projects => generated 367 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:46:08.657354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3cc48-c59f-4fae-8e65-e9dfff7d7ecd None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:08.668302 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3cc48-c59f-4fae-8e65-e9dfff7d7ecd None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:08.724546 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db0528-8fb7-4601-a9bd-08cc2cd020df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:08.725254 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db0528-8fb7-4601-a9bd-08cc2cd020df admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:08.725538 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db0528-8fb7-4601-a9bd-08cc2cd020df admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:08.725815 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db0528-8fb7-4601-a9bd-08cc2cd020df admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:08.726499 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db0528-8fb7-4601-a9bd-08cc2cd020df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:08.728436 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db0528-8fb7-4601-a9bd-08cc2cd020df admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:08.777128 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1324/2676] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:46:08 2022] POST /identity/v3/users => generated 290 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:46:08.781627 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96258979-57ed-499f-9c9c-c185ca6e9c53 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:08.791970 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96258979-57ed-499f-9c9c-c185ca6e9c53 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:08.796830 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcef7a4-df86-467f-8104-80cdffadfbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:08.797522 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcef7a4-df86-467f-8104-80cdffadfbec admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:08.797847 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcef7a4-df86-467f-8104-80cdffadfbec admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:08.798159 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcef7a4-df86-467f-8104-80cdffadfbec admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:08.798896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcef7a4-df86-467f-8104-80cdffadfbec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:08.800497 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcef7a4-df86-467f-8104-80cdffadfbec admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:08.811093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1353/2677] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:46:08.816326 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db0528-8fb7-4601-a9bd-08cc2cd020df admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:08.826386 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db0528-8fb7-4601-a9bd-08cc2cd020df admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:08.831629 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34bbe2-bc58-42ca-9410-df452bb7ca1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:08.832517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34bbe2-bc58-42ca-9410-df452bb7ca1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:08.832792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34bbe2-bc58-42ca-9410-df452bb7ca1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:08.833115 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34bbe2-bc58-42ca-9410-df452bb7ca1b admin admin] PATH_INFO: `/v3/projects/4e8f8820dcdc4b6e90bb2fa4073fa0c7/users/12eeec55ecd24f4187cdccd1707d607f/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:08.847862 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34bbe2-bc58-42ca-9410-df452bb7ca1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8f8820dcdc4b6e90bb2fa4073fa0c7, user_id=12eeec55ecd24f4187cdccd1707d607f, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:08.850526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34bbe2-bc58-42ca-9410-df452bb7ca1b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:08.955757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1325/2678] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:46:08 2022] PUT /identity/v3/projects/4e8f8820dcdc4b6e90bb2fa4073fa0c7/users/12eeec55ecd24f4187cdccd1707d607f/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:08.960308 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcef7a4-df86-467f-8104-80cdffadfbec admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:08.970702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcef7a4-df86-467f-8104-80cdffadfbec admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:09.029182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e3f34-e715-4361-83ee-6f934a46dd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:09.029852 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e3f34-e715-4361-83ee-6f934a46dd57 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:09.030134 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e3f34-e715-4361-83ee-6f934a46dd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:09.030533 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e3f34-e715-4361-83ee-6f934a46dd57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:09.031267 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e3f34-e715-4361-83ee-6f934a46dd57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:09.032967 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e3f34-e715-4361-83ee-6f934a46dd57 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:09.043136 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1354/2679] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:46:09.047559 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34bbe2-bc58-42ca-9410-df452bb7ca1b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:09.059462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34bbe2-bc58-42ca-9410-df452bb7ca1b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:09.064939 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c918656-e816-4c03-a178-b4c93f06c39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:09.065908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c918656-e816-4c03-a178-b4c93f06c39a admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:09.068365 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c918656-e816-4c03-a178-b4c93f06c39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:09.068365 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c918656-e816-4c03-a178-b4c93f06c39a admin admin] PATH_INFO: `/v3/projects/4e8f8820dcdc4b6e90bb2fa4073fa0c7/users/12eeec55ecd24f4187cdccd1707d607f/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:09.069006 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c918656-e816-4c03-a178-b4c93f06c39a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8f8820dcdc4b6e90bb2fa4073fa0c7, user_id=12eeec55ecd24f4187cdccd1707d607f, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:09.071814 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c918656-e816-4c03-a178-b4c93f06c39a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:09.083034 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1326/2680] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:46:09 2022] PUT /identity/v3/projects/4e8f8820dcdc4b6e90bb2fa4073fa0c7/users/12eeec55ecd24f4187cdccd1707d607f/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:09.093512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c687f-ab84-4148-968a-570f6e39bb31 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:09.093805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c687f-ab84-4148-968a-570f6e39bb31 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:09.094089 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c687f-ab84-4148-968a-570f6e39bb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:09.121742 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-642c687f-ab84-4148-968a-570f6e39bb31 None None] MFA Rules not processed for user `12eeec55ecd24f4187cdccd1707d607f`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:09.122975 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-642c687f-ab84-4148-968a-570f6e39bb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:09.198295 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1355/2681] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:09 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:46:09.208908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c918656-e816-4c03-a178-b4c93f06c39a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:09.210747 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8c918656-e816-4c03-a178-b4c93f06c39a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:09.270121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c918656-e816-4c03-a178-b4c93f06c39a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:09.273745 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca486b5d-da81-4c3d-95ab-6db49c600e9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:09.274391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca486b5d-da81-4c3d-95ab-6db49c600e9c service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:09.274672 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca486b5d-da81-4c3d-95ab-6db49c600e9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:09.274942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca486b5d-da81-4c3d-95ab-6db49c600e9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:09.284577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca486b5d-da81-4c3d-95ab-6db49c600e9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:09.286076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca486b5d-da81-4c3d-95ab-6db49c600e9c service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:09.295661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1327/2682] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:46:09 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:46:11.886387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c687f-ab84-4148-968a-570f6e39bb31 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:11.928510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:11.950088 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c687f-ab84-4148-968a-570f6e39bb31 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:11.955223 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45884ca-7daa-45ef-891c-f5d26062fc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:11.956087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45884ca-7daa-45ef-891c-f5d26062fc55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:11.956364 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45884ca-7daa-45ef-891c-f5d26062fc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:11.956633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45884ca-7daa-45ef-891c-f5d26062fc55 admin admin] PATH_INFO: `/v3/users/c1f38e2933c449189a221be793e55aee` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:11.957827 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45884ca-7daa-45ef-891c-f5d26062fc55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1f38e2933c449189a221be793e55aee)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:11.960208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45884ca-7daa-45ef-891c-f5d26062fc55 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:12.130048 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d45884ca-7daa-45ef-891c-f5d26062fc55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1f38e2933c449189a221be793e55aee'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:12.130902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d45884ca-7daa-45ef-891c-f5d26062fc55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1f38e2933c449189a221be793e55aee'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:12.138571 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d45884ca-7daa-45ef-891c-f5d26062fc55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1f38e2933c449189a221be793e55aee'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:12.139333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d45884ca-7daa-45ef-891c-f5d26062fc55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1f38e2933c449189a221be793e55aee'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:12.194723 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d45884ca-7daa-45ef-891c-f5d26062fc55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1f38e2933c449189a221be793e55aee'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:12.217904 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1356/2683] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:46:11 2022] DELETE /identity/v3/users/c1f38e2933c449189a221be793e55aee => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:12.222848 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e4737-6523-404d-b741-598fe57fa11a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:12.223439 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e4737-6523-404d-b741-598fe57fa11a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:12.223879 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e4737-6523-404d-b741-598fe57fa11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:12.246949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b20e4737-6523-404d-b741-598fe57fa11a None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:12.248209 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b20e4737-6523-404d-b741-598fe57fa11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:12.285636 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:12.318854 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1328/2684] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:46:12.330400 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45884ca-7daa-45ef-891c-f5d26062fc55 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:12.332142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d45884ca-7daa-45ef-891c-f5d26062fc55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:12.397074 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45884ca-7daa-45ef-891c-f5d26062fc55 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:12.401157 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a210a6b-66e1-4968-b654-63c21c427d9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:12.401841 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a210a6b-66e1-4968-b654-63c21c427d9c service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:12.402128 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a210a6b-66e1-4968-b654-63c21c427d9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:12.402410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a210a6b-66e1-4968-b654-63c21c427d9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:12.413074 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a210a6b-66e1-4968-b654-63c21c427d9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:12.414597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a210a6b-66e1-4968-b654-63c21c427d9c service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:12.425177 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1357/2685] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:46:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:46:12.814612 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e4737-6523-404d-b741-598fe57fa11a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:12.817041 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b20e4737-6523-404d-b741-598fe57fa11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:12.841907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e4737-6523-404d-b741-598fe57fa11a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:12.846830 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047fb10-3221-4bd1-9f9b-b99d9113e8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:12.847766 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047fb10-3221-4bd1-9f9b-b99d9113e8df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:12.848079 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047fb10-3221-4bd1-9f9b-b99d9113e8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:12.848365 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047fb10-3221-4bd1-9f9b-b99d9113e8df admin admin] PATH_INFO: `/v3/projects/77a71fdc2a8341859a1d45f3f53f3301` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:12.849568 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047fb10-3221-4bd1-9f9b-b99d9113e8df admin admin] RBAC: Authorizing `identity:delete_project(project_id=77a71fdc2a8341859a1d45f3f53f3301)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:12.851592 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047fb10-3221-4bd1-9f9b-b99d9113e8df admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:13.009040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8047fb10-3221-4bd1-9f9b-b99d9113e8df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77a71fdc2a8341859a1d45f3f53f3301', 'request_id': 'req-8047fb10-3221-4bd1-9f9b-b99d9113e8df'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:13.014499 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8047fb10-3221-4bd1-9f9b-b99d9113e8df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77a71fdc2a8341859a1d45f3f53f3301', 'request_id': 'req-8047fb10-3221-4bd1-9f9b-b99d9113e8df'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:13.020854 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8047fb10-3221-4bd1-9f9b-b99d9113e8df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77a71fdc2a8341859a1d45f3f53f3301', 'request_id': 'req-8047fb10-3221-4bd1-9f9b-b99d9113e8df'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:13.131439 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8047fb10-3221-4bd1-9f9b-b99d9113e8df admin admin] The token cache is being invalidate because project 77a71fdc2a8341859a1d45f3f53f3301 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:46:13.131733 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8047fb10-3221-4bd1-9f9b-b99d9113e8df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:13.132395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8047fb10-3221-4bd1-9f9b-b99d9113e8df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:13.134141 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1329/2686] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:46:12 2022] DELETE /identity/v3/projects/77a71fdc2a8341859a1d45f3f53f3301 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:13.140367 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a210a6b-66e1-4968-b654-63c21c427d9c service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:13.148101 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a210a6b-66e1-4968-b654-63c21c427d9c service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:13.196268 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047fb10-3221-4bd1-9f9b-b99d9113e8df admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:13.198103 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8047fb10-3221-4bd1-9f9b-b99d9113e8df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:13.228565 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc30d0-b637-442e-a2b6-61e0b9bc8f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:13.229990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc30d0-b637-442e-a2b6-61e0b9bc8f3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:13.230456 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc30d0-b637-442e-a2b6-61e0b9bc8f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:13.230927 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc30d0-b637-442e-a2b6-61e0b9bc8f3f admin admin] PATH_INFO: `/v3/users/12eeec55ecd24f4187cdccd1707d607f` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:13.232908 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc30d0-b637-442e-a2b6-61e0b9bc8f3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=12eeec55ecd24f4187cdccd1707d607f)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:13.236918 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc30d0-b637-442e-a2b6-61e0b9bc8f3f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:13.239315 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:13.281776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047fb10-3221-4bd1-9f9b-b99d9113e8df admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:13.287352 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347e1f5-8fa8-41f4-a41b-cf34028d579b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc0d7bd7e4547f4a8d90ffc2c2e1d03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:13.288005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347e1f5-8fa8-41f4-a41b-cf34028d579b service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:13.288289 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347e1f5-8fa8-41f4-a41b-cf34028d579b service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:13.288564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347e1f5-8fa8-41f4-a41b-cf34028d579b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:13.290478 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d347e1f5-8fa8-41f4-a41b-cf34028d579b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:13.303342 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347e1f5-8fa8-41f4-a41b-cf34028d579b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:13.304984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347e1f5-8fa8-41f4-a41b-cf34028d579b service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:13.315447 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1330/2687] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:46:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:46:13.441112 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-80fc30d0-b637-442e-a2b6-61e0b9bc8f3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12eeec55ecd24f4187cdccd1707d607f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:13.441768 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-80fc30d0-b637-442e-a2b6-61e0b9bc8f3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12eeec55ecd24f4187cdccd1707d607f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:13.448415 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-80fc30d0-b637-442e-a2b6-61e0b9bc8f3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12eeec55ecd24f4187cdccd1707d607f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:13.449088 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-80fc30d0-b637-442e-a2b6-61e0b9bc8f3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12eeec55ecd24f4187cdccd1707d607f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:13.543556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-80fc30d0-b637-442e-a2b6-61e0b9bc8f3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12eeec55ecd24f4187cdccd1707d607f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:13.563623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1358/2688] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:46:13 2022] DELETE /identity/v3/users/12eeec55ecd24f4187cdccd1707d607f => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:46:13.910543 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347e1f5-8fa8-41f4-a41b-cf34028d579b service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:13.912274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d347e1f5-8fa8-41f4-a41b-cf34028d579b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:13.929055 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347e1f5-8fa8-41f4-a41b-cf34028d579b service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:13.977721 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:13.987530 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab7b72-3a75-4ea0-a795-d6f5851f0738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:13.988391 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab7b72-3a75-4ea0-a795-d6f5851f0738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:13.988697 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab7b72-3a75-4ea0-a795-d6f5851f0738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:13.988971 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab7b72-3a75-4ea0-a795-d6f5851f0738 admin admin] PATH_INFO: `/v3/projects/4e8f8820dcdc4b6e90bb2fa4073fa0c7` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:13.990161 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab7b72-3a75-4ea0-a795-d6f5851f0738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e8f8820dcdc4b6e90bb2fa4073fa0c7)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:13.992071 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab7b72-3a75-4ea0-a795-d6f5851f0738 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:14.100029 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ceab7b72-3a75-4ea0-a795-d6f5851f0738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e8f8820dcdc4b6e90bb2fa4073fa0c7', 'request_id': 'req-ceab7b72-3a75-4ea0-a795-d6f5851f0738'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:14.105276 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ceab7b72-3a75-4ea0-a795-d6f5851f0738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e8f8820dcdc4b6e90bb2fa4073fa0c7', 'request_id': 'req-ceab7b72-3a75-4ea0-a795-d6f5851f0738'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:14.111622 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ceab7b72-3a75-4ea0-a795-d6f5851f0738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e8f8820dcdc4b6e90bb2fa4073fa0c7', 'request_id': 'req-ceab7b72-3a75-4ea0-a795-d6f5851f0738'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:14.196521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ceab7b72-3a75-4ea0-a795-d6f5851f0738 admin admin] The token cache is being invalidate because project 4e8f8820dcdc4b6e90bb2fa4073fa0c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:46:14.196814 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ceab7b72-3a75-4ea0-a795-d6f5851f0738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:14.197522 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ceab7b72-3a75-4ea0-a795-d6f5851f0738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:14.199528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1331/2689] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:46:13 2022] DELETE /identity/v3/projects/4e8f8820dcdc4b6e90bb2fa4073fa0c7 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:14.205861 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89b396-eacf-411f-ba92-6872dceddfab None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:14.206433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89b396-eacf-411f-ba92-6872dceddfab None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:14.206433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89b396-eacf-411f-ba92-6872dceddfab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:14.225648 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-4a89b396-eacf-411f-ba92-6872dceddfab None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:14.227243 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4a89b396-eacf-411f-ba92-6872dceddfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:14.289579 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1359/2690] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:46:14.298258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cf273-df8e-4ae6-be85-b6149e6cb16d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:14.298979 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cf273-df8e-4ae6-be85-b6149e6cb16d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:14.298979 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cf273-df8e-4ae6-be85-b6149e6cb16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:14.318184 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d60cf273-df8e-4ae6-be85-b6149e6cb16d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:14.319406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d60cf273-df8e-4ae6-be85-b6149e6cb16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:14.327632 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1332/2691] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:46:14.331985 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89b396-eacf-411f-ba92-6872dceddfab None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:14.341987 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89b396-eacf-411f-ba92-6872dceddfab None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:14.346649 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d8929-7f2b-4690-9497-b3b919bd987f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:14.347374 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d8929-7f2b-4690-9497-b3b919bd987f admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:14.347667 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d8929-7f2b-4690-9497-b3b919bd987f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:14.347943 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d8929-7f2b-4690-9497-b3b919bd987f admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:14.348671 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d8929-7f2b-4690-9497-b3b919bd987f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:46:14.348980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d8929-7f2b-4690-9497-b3b919bd987f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:14.350670 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d8929-7f2b-4690-9497-b3b919bd987f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:14.364280 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1360/2692] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:46:14.369163 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cf273-df8e-4ae6-be85-b6149e6cb16d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:14.379120 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cf273-df8e-4ae6-be85-b6149e6cb16d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:14.383958 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c34aa2-c51d-4f8f-96ea-499b64681ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:14.384611 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c34aa2-c51d-4f8f-96ea-499b64681ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:14.384893 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c34aa2-c51d-4f8f-96ea-499b64681ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:14.385198 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c34aa2-c51d-4f8f-96ea-499b64681ff6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:14.385894 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c34aa2-c51d-4f8f-96ea-499b64681ff6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:14.387850 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c34aa2-c51d-4f8f-96ea-499b64681ff6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:14.464076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1333/2693] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:46:14 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:46:14.468683 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d8929-7f2b-4690-9497-b3b919bd987f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:14.479654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d8929-7f2b-4690-9497-b3b919bd987f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:14.537958 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f368a81-0550-47ea-84f2-326c756453fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:14.538625 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f368a81-0550-47ea-84f2-326c756453fb admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:14.538907 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f368a81-0550-47ea-84f2-326c756453fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:14.539206 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f368a81-0550-47ea-84f2-326c756453fb admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:14.539869 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f368a81-0550-47ea-84f2-326c756453fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:14.541835 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f368a81-0550-47ea-84f2-326c756453fb admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:14.552051 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:14.663877 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1361/2694] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:46:14 2022] POST /identity/v3/users => generated 294 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:46:14.667989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c34aa2-c51d-4f8f-96ea-499b64681ff6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:14.679063 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c34aa2-c51d-4f8f-96ea-499b64681ff6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:14.683805 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b392c-9678-47b5-b587-49a4fccd8e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:14.684462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b392c-9678-47b5-b587-49a4fccd8e48 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:14.684750 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b392c-9678-47b5-b587-49a4fccd8e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:14.685070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b392c-9678-47b5-b587-49a4fccd8e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:14.685799 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b392c-9678-47b5-b587-49a4fccd8e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:14.687393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b392c-9678-47b5-b587-49a4fccd8e48 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:14.697934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1334/2695] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:46:14.702188 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f368a81-0550-47ea-84f2-326c756453fb admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:14.712524 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f368a81-0550-47ea-84f2-326c756453fb admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:14.717947 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f642e4f-806a-4f7c-b97f-9932a1d85daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:14.718638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f642e4f-806a-4f7c-b97f-9932a1d85daa admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:14.718916 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f642e4f-806a-4f7c-b97f-9932a1d85daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:14.719189 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f642e4f-806a-4f7c-b97f-9932a1d85daa admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:14.720503 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f642e4f-806a-4f7c-b97f-9932a1d85daa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:14.722168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f642e4f-806a-4f7c-b97f-9932a1d85daa admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:14.732213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1362/2696] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:46:14.737277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b392c-9678-47b5-b587-49a4fccd8e48 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:14.747213 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b392c-9678-47b5-b587-49a4fccd8e48 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:14.752103 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe470b0-5fdb-4253-90a2-751ff154cde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:14.752982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe470b0-5fdb-4253-90a2-751ff154cde3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:14.753304 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe470b0-5fdb-4253-90a2-751ff154cde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:14.753578 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe470b0-5fdb-4253-90a2-751ff154cde3 admin admin] PATH_INFO: `/v3/projects/e35ec6ee77df4106850c696f39fa0641/users/fef138a7938e4598af4962eca3238943/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:14.768065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe470b0-5fdb-4253-90a2-751ff154cde3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e35ec6ee77df4106850c696f39fa0641, user_id=fef138a7938e4598af4962eca3238943, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:14.770781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe470b0-5fdb-4253-90a2-751ff154cde3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:14.840032 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1335/2697] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:46:14 2022] PUT /identity/v3/projects/e35ec6ee77df4106850c696f39fa0641/users/fef138a7938e4598af4962eca3238943/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:14.859679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf198f-0360-4bed-a406-dfdf09ffcc42 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:14.860620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf198f-0360-4bed-a406-dfdf09ffcc42 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:14.861160 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf198f-0360-4bed-a406-dfdf09ffcc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:14.891222 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-daaf198f-0360-4bed-a406-dfdf09ffcc42 None None] MFA Rules not processed for user `fef138a7938e4598af4962eca3238943`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:14.892521 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-daaf198f-0360-4bed-a406-dfdf09ffcc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:14.968693 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1363/2698] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:46:14.974762 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe470b0-5fdb-4253-90a2-751ff154cde3 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:15.046072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe470b0-5fdb-4253-90a2-751ff154cde3 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:15.051213 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ea79a-4b9f-45b4-b69f-77d2f835cebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:15.051868 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ea79a-4b9f-45b4-b69f-77d2f835cebb admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:15.052150 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ea79a-4b9f-45b4-b69f-77d2f835cebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:15.052424 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ea79a-4b9f-45b4-b69f-77d2f835cebb admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:15.053156 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ea79a-4b9f-45b4-b69f-77d2f835cebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:15.055158 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ea79a-4b9f-45b4-b69f-77d2f835cebb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:15.167549 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1336/2699] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:46:14 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:46:15.171388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf198f-0360-4bed-a406-dfdf09ffcc42 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:15.184217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf198f-0360-4bed-a406-dfdf09ffcc42 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:15.243691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a245c6-7301-421b-8b14-8396bcd66de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:15.244740 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a245c6-7301-421b-8b14-8396bcd66de0 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:15.244913 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a245c6-7301-421b-8b14-8396bcd66de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:15.245177 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a245c6-7301-421b-8b14-8396bcd66de0 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:15.245884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a245c6-7301-421b-8b14-8396bcd66de0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:15.247877 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a245c6-7301-421b-8b14-8396bcd66de0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:15.355173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1364/2700] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:46:15 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:46:15.359678 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ea79a-4b9f-45b4-b69f-77d2f835cebb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:15.366194 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:15.371680 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ea79a-4b9f-45b4-b69f-77d2f835cebb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:15.376414 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db30e88-3e7d-4a1a-8089-d2b3610e98c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:15.377189 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db30e88-3e7d-4a1a-8089-d2b3610e98c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:15.379413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db30e88-3e7d-4a1a-8089-d2b3610e98c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:15.379413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db30e88-3e7d-4a1a-8089-d2b3610e98c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:15.379413 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db30e88-3e7d-4a1a-8089-d2b3610e98c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:15.380092 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db30e88-3e7d-4a1a-8089-d2b3610e98c8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:15.391235 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1337/2701] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:46:15.395543 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a245c6-7301-421b-8b14-8396bcd66de0 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:15.407557 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a245c6-7301-421b-8b14-8396bcd66de0 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:15.412307 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96723bf3-3289-4788-960c-d099bc5e20ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:15.413022 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96723bf3-3289-4788-960c-d099bc5e20ff admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:15.413303 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96723bf3-3289-4788-960c-d099bc5e20ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:15.413602 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96723bf3-3289-4788-960c-d099bc5e20ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:15.414312 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96723bf3-3289-4788-960c-d099bc5e20ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:15.415914 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96723bf3-3289-4788-960c-d099bc5e20ff admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:15.427212 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1365/2702] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:46:15.431529 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db30e88-3e7d-4a1a-8089-d2b3610e98c8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:15.441735 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db30e88-3e7d-4a1a-8089-d2b3610e98c8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:15.446572 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c2f86-c1fb-403f-a244-4d9e21ebe197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:15.447611 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c2f86-c1fb-403f-a244-4d9e21ebe197 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:15.447933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c2f86-c1fb-403f-a244-4d9e21ebe197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:15.448206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c2f86-c1fb-403f-a244-4d9e21ebe197 admin admin] PATH_INFO: `/v3/projects/aafb47ec66f346abbdb27b52b140946a/users/34c1a06881fd429383200eb2a0a74fd3/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:15.474597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c2f86-c1fb-403f-a244-4d9e21ebe197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aafb47ec66f346abbdb27b52b140946a, user_id=34c1a06881fd429383200eb2a0a74fd3, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:15.480007 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c2f86-c1fb-403f-a244-4d9e21ebe197 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:15.549807 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1338/2703] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:46:15 2022] PUT /identity/v3/projects/aafb47ec66f346abbdb27b52b140946a/users/34c1a06881fd429383200eb2a0a74fd3/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:15.560990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e1eaa-85e1-4238-926c-6f84a0116d0c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:15.561639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e1eaa-85e1-4238-926c-6f84a0116d0c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:15.561894 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e1eaa-85e1-4238-926c-6f84a0116d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:15.591499 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9d0e1eaa-85e1-4238-926c-6f84a0116d0c None None] MFA Rules not processed for user `34c1a06881fd429383200eb2a0a74fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:15.592837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9d0e1eaa-85e1-4238-926c-6f84a0116d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:15.636540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:15.659900 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1366/2704] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:46:15.664505 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c2f86-c1fb-403f-a244-4d9e21ebe197 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:15.728405 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c2f86-c1fb-403f-a244-4d9e21ebe197 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:15.733148 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aafad5-562f-4a1e-bbbc-827dccbf68c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:15.733838 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aafad5-562f-4a1e-bbbc-827dccbf68c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:15.734093 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aafad5-562f-4a1e-bbbc-827dccbf68c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:15.734449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aafad5-562f-4a1e-bbbc-827dccbf68c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:15.735191 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aafad5-562f-4a1e-bbbc-827dccbf68c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:15.737147 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aafad5-562f-4a1e-bbbc-827dccbf68c4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:15.818507 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1339/2705] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:46:15 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:46:15.822596 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e1eaa-85e1-4238-926c-6f84a0116d0c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:15.834092 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e1eaa-85e1-4238-926c-6f84a0116d0c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:15.889303 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02d9af-34e2-4038-bf71-33849b8b93b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:15.890042 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02d9af-34e2-4038-bf71-33849b8b93b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:15.890325 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02d9af-34e2-4038-bf71-33849b8b93b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:15.890599 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02d9af-34e2-4038-bf71-33849b8b93b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:15.891812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02d9af-34e2-4038-bf71-33849b8b93b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:15.893810 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02d9af-34e2-4038-bf71-33849b8b93b5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:15.957054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1367/2706] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:46:15 2022] POST /identity/v3/users => generated 298 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:46:15.959124 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aafad5-562f-4a1e-bbbc-827dccbf68c4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:15.970086 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aafad5-562f-4a1e-bbbc-827dccbf68c4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:15.974969 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070f999-91cf-4802-834d-3ec836d3adc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:15.975633 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070f999-91cf-4802-834d-3ec836d3adc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:15.975912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070f999-91cf-4802-834d-3ec836d3adc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:15.976328 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070f999-91cf-4802-834d-3ec836d3adc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:15.976911 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a070f999-91cf-4802-834d-3ec836d3adc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:15.978582 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a070f999-91cf-4802-834d-3ec836d3adc1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:15.989655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1340/2707] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:46:15.994067 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02d9af-34e2-4038-bf71-33849b8b93b5 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:16.004568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02d9af-34e2-4038-bf71-33849b8b93b5 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:16.009330 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ace2a-b015-4cc4-a0b5-c82e1515aafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:16.010201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ace2a-b015-4cc4-a0b5-c82e1515aafd admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:16.013421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ace2a-b015-4cc4-a0b5-c82e1515aafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:16.013421 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ace2a-b015-4cc4-a0b5-c82e1515aafd admin admin] PATH_INFO: `/v3/projects/add3cd4351ae4bedaf4e48c04f65b2fe/users/03fe64b4adb5415fb453d2c8d85566bd/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:16.026695 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ace2a-b015-4cc4-a0b5-c82e1515aafd admin admin] RBAC: Authorizing `identity:create_grant(project_id=add3cd4351ae4bedaf4e48c04f65b2fe, user_id=03fe64b4adb5415fb453d2c8d85566bd, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:16.030667 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ace2a-b015-4cc4-a0b5-c82e1515aafd admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:16.108380 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1368/2708] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:46:15 2022] PUT /identity/v3/projects/add3cd4351ae4bedaf4e48c04f65b2fe/users/03fe64b4adb5415fb453d2c8d85566bd/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:16.112539 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070f999-91cf-4802-834d-3ec836d3adc1 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:16.122656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070f999-91cf-4802-834d-3ec836d3adc1 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:16.182289 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63733ae7-ae31-42f8-89e2-5040e604c605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:16.191920 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63733ae7-ae31-42f8-89e2-5040e604c605 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:16.191920 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63733ae7-ae31-42f8-89e2-5040e604c605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:16.191920 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63733ae7-ae31-42f8-89e2-5040e604c605 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:16.191920 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63733ae7-ae31-42f8-89e2-5040e604c605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:16.191920 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63733ae7-ae31-42f8-89e2-5040e604c605 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:16.198132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1341/2709] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:46:16.205384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ace2a-b015-4cc4-a0b5-c82e1515aafd admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:16.218588 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ace2a-b015-4cc4-a0b5-c82e1515aafd admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:16.224252 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba4e75-f846-4730-b277-67f123370955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:16.225337 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba4e75-f846-4730-b277-67f123370955 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:16.225739 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba4e75-f846-4730-b277-67f123370955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:16.226134 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba4e75-f846-4730-b277-67f123370955 admin admin] PATH_INFO: `/v3/projects/add3cd4351ae4bedaf4e48c04f65b2fe/users/03fe64b4adb5415fb453d2c8d85566bd/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:16.229470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba4e75-f846-4730-b277-67f123370955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=add3cd4351ae4bedaf4e48c04f65b2fe, user_id=03fe64b4adb5415fb453d2c8d85566bd, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:16.233063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba4e75-f846-4730-b277-67f123370955 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:16.245168 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1369/2710] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:46:16 2022] PUT /identity/v3/projects/add3cd4351ae4bedaf4e48c04f65b2fe/users/03fe64b4adb5415fb453d2c8d85566bd/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:16.256852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac20d65-9ec4-4c09-9b5d-d3b5e9de6cc8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:16.257199 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac20d65-9ec4-4c09-9b5d-d3b5e9de6cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:16.257444 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac20d65-9ec4-4c09-9b5d-d3b5e9de6cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:16.285737 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2ac20d65-9ec4-4c09-9b5d-d3b5e9de6cc8 None None] MFA Rules not processed for user `03fe64b4adb5415fb453d2c8d85566bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:16.286947 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2ac20d65-9ec4-4c09-9b5d-d3b5e9de6cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:16.312594 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:16.354741 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1342/2711] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:46:16.365841 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba4e75-f846-4730-b277-67f123370955 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:16.367573 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-78ba4e75-f846-4730-b277-67f123370955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:16.443185 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba4e75-f846-4730-b277-67f123370955 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:16.446811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ef88d-902d-4cda-a32d-e5a2902def5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:16.447451 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ef88d-902d-4cda-a32d-e5a2902def5b service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:16.447734 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ef88d-902d-4cda-a32d-e5a2902def5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:16.448004 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ef88d-902d-4cda-a32d-e5a2902def5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:16.457495 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ef88d-902d-4cda-a32d-e5a2902def5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:16.458918 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ef88d-902d-4cda-a32d-e5a2902def5b service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:16.468084 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1370/2712] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:46:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:46:17.530781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac20d65-9ec4-4c09-9b5d-d3b5e9de6cc8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:17.532796 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2ac20d65-9ec4-4c09-9b5d-d3b5e9de6cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:17.545548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac20d65-9ec4-4c09-9b5d-d3b5e9de6cc8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:17.549334 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ea36b-74a8-4533-97c6-b7760855b4f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:17.549977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ea36b-74a8-4533-97c6-b7760855b4f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:17.550254 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ea36b-74a8-4533-97c6-b7760855b4f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:17.550525 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ea36b-74a8-4533-97c6-b7760855b4f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:17.618291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ea36b-74a8-4533-97c6-b7760855b4f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:17.619780 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ea36b-74a8-4533-97c6-b7760855b4f6 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:17.629867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1343/2713] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:46:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:46:22.608592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ef88d-902d-4cda-a32d-e5a2902def5b service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:22.618345 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ef88d-902d-4cda-a32d-e5a2902def5b service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:22.622213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6d6e8-e2c4-487b-8230-dc95afa72161 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:22.622909 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6d6e8-e2c4-487b-8230-dc95afa72161 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:22.623410 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6d6e8-e2c4-487b-8230-dc95afa72161 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:22.623775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6d6e8-e2c4-487b-8230-dc95afa72161 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:22.665308 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:22.688715 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6d6e8-e2c4-487b-8230-dc95afa72161 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:22.690205 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6d6e8-e2c4-487b-8230-dc95afa72161 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:22.699656 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1371/2714] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:46:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:46:26.926639 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ea36b-74a8-4533-97c6-b7760855b4f6 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:26.997967 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ea36b-74a8-4533-97c6-b7760855b4f6 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:27.003417 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025438b-2bfc-4da6-a28d-b80623e123e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:27.004440 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025438b-2bfc-4da6-a28d-b80623e123e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:27.004726 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025438b-2bfc-4da6-a28d-b80623e123e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:27.005025 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025438b-2bfc-4da6-a28d-b80623e123e8 admin admin] PATH_INFO: `/v3/users/fef138a7938e4598af4962eca3238943` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:27.006217 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025438b-2bfc-4da6-a28d-b80623e123e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fef138a7938e4598af4962eca3238943)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:27.008136 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025438b-2bfc-4da6-a28d-b80623e123e8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:27.216363 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1025438b-2bfc-4da6-a28d-b80623e123e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fef138a7938e4598af4962eca3238943'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:27.217230 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1025438b-2bfc-4da6-a28d-b80623e123e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fef138a7938e4598af4962eca3238943'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:27.224753 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1025438b-2bfc-4da6-a28d-b80623e123e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fef138a7938e4598af4962eca3238943'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:27.225453 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1025438b-2bfc-4da6-a28d-b80623e123e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fef138a7938e4598af4962eca3238943'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:27.232082 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:27.240263 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1025438b-2bfc-4da6-a28d-b80623e123e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fef138a7938e4598af4962eca3238943'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:27.360368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1344/2715] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:46:26 2022] DELETE /identity/v3/users/fef138a7938e4598af4962eca3238943 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:27.365580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be5ca5-1eb2-4f6d-9bd0-a4c43a10fc24 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:27.366401 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be5ca5-1eb2-4f6d-9bd0-a4c43a10fc24 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:27.366588 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be5ca5-1eb2-4f6d-9bd0-a4c43a10fc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:27.382182 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9d345-e435-4ab0-8901-d44be6af9b8c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:27.382472 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9d345-e435-4ab0-8901-d44be6af9b8c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:27.382745 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9d345-e435-4ab0-8901-d44be6af9b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:27.391919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-61be5ca5-1eb2-4f6d-9bd0-a4c43a10fc24 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:27.393977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-61be5ca5-1eb2-4f6d-9bd0-a4c43a10fc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:27.407874 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5fe9d345-e435-4ab0-8901-d44be6af9b8c None None] MFA Rules not processed for user `52ddf59fe764404eb1c6ee745e21d352`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:27.410904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5fe9d345-e435-4ab0-8901-d44be6af9b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:27.472007 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1372/2716] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:46:27.482586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1345/2717] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:46:27.483175 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be5ca5-1eb2-4f6d-9bd0-a4c43a10fc24 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:27.484566 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-61be5ca5-1eb2-4f6d-9bd0-a4c43a10fc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:27.548711 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be5ca5-1eb2-4f6d-9bd0-a4c43a10fc24 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:27.550954 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80ebe5-12f1-4c0a-9813-801cb281dd2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:27.551711 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80ebe5-12f1-4c0a-9813-801cb281dd2d service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:27.552647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80ebe5-12f1-4c0a-9813-801cb281dd2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:27.553051 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80ebe5-12f1-4c0a-9813-801cb281dd2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:27.563748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80ebe5-12f1-4c0a-9813-801cb281dd2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:27.565232 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80ebe5-12f1-4c0a-9813-801cb281dd2d service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:27.576134 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1373/2718] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:46:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:46:28.498608 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9d345-e435-4ab0-8901-d44be6af9b8c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:28.501223 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5fe9d345-e435-4ab0-8901-d44be6af9b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:28.515274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9d345-e435-4ab0-8901-d44be6af9b8c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:28.520286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16e878-a887-49aa-8c99-812a82496232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:28.521272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16e878-a887-49aa-8c99-812a82496232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:28.521557 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16e878-a887-49aa-8c99-812a82496232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:28.521867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16e878-a887-49aa-8c99-812a82496232 admin admin] PATH_INFO: `/v3/projects/e35ec6ee77df4106850c696f39fa0641` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:28.523048 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16e878-a887-49aa-8c99-812a82496232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e35ec6ee77df4106850c696f39fa0641)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:28.525036 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16e878-a887-49aa-8c99-812a82496232 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:28.653887 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cb16e878-a887-49aa-8c99-812a82496232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e35ec6ee77df4106850c696f39fa0641', 'request_id': 'req-cb16e878-a887-49aa-8c99-812a82496232'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:28.669226 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cb16e878-a887-49aa-8c99-812a82496232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e35ec6ee77df4106850c696f39fa0641', 'request_id': 'req-cb16e878-a887-49aa-8c99-812a82496232'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:28.678600 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cb16e878-a887-49aa-8c99-812a82496232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e35ec6ee77df4106850c696f39fa0641', 'request_id': 'req-cb16e878-a887-49aa-8c99-812a82496232'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:28.749723 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cb16e878-a887-49aa-8c99-812a82496232 admin admin] The token cache is being invalidate because project e35ec6ee77df4106850c696f39fa0641 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:46:28.750012 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cb16e878-a887-49aa-8c99-812a82496232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:28.751242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cb16e878-a887-49aa-8c99-812a82496232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:28.753014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1346/2719] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:46:28 2022] DELETE /identity/v3/projects/e35ec6ee77df4106850c696f39fa0641 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:28.761054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80ebe5-12f1-4c0a-9813-801cb281dd2d service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:28.772558 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80ebe5-12f1-4c0a-9813-801cb281dd2d service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:28.836427 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44173c-5c0f-4263-80b0-25e8d733c4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:28.837455 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44173c-5c0f-4263-80b0-25e8d733c4d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:28.837734 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44173c-5c0f-4263-80b0-25e8d733c4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:28.838006 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44173c-5c0f-4263-80b0-25e8d733c4d6 admin admin] PATH_INFO: `/v3/users/34c1a06881fd429383200eb2a0a74fd3` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:28.839409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44173c-5c0f-4263-80b0-25e8d733c4d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34c1a06881fd429383200eb2a0a74fd3)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:28.841756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44173c-5c0f-4263-80b0-25e8d733c4d6 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:28.866544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:29.096751 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8c44173c-5c0f-4263-80b0-25e8d733c4d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34c1a06881fd429383200eb2a0a74fd3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:29.097594 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8c44173c-5c0f-4263-80b0-25e8d733c4d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34c1a06881fd429383200eb2a0a74fd3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:29.105923 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8c44173c-5c0f-4263-80b0-25e8d733c4d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34c1a06881fd429383200eb2a0a74fd3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:29.106639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8c44173c-5c0f-4263-80b0-25e8d733c4d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34c1a06881fd429383200eb2a0a74fd3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:29.205353 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8c44173c-5c0f-4263-80b0-25e8d733c4d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34c1a06881fd429383200eb2a0a74fd3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:29.228448 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1374/2720] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:46:28 2022] DELETE /identity/v3/users/34c1a06881fd429383200eb2a0a74fd3 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:29.688353 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16e878-a887-49aa-8c99-812a82496232 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:29.690849 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cb16e878-a887-49aa-8c99-812a82496232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:29.711339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16e878-a887-49aa-8c99-812a82496232 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:29.783643 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c811e-5627-4e6b-83da-7f58cd54c4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:29.784565 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c811e-5627-4e6b-83da-7f58cd54c4d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:29.784849 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c811e-5627-4e6b-83da-7f58cd54c4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:29.785191 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c811e-5627-4e6b-83da-7f58cd54c4d0 admin admin] PATH_INFO: `/v3/projects/aafb47ec66f346abbdb27b52b140946a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:29.786382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c811e-5627-4e6b-83da-7f58cd54c4d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aafb47ec66f346abbdb27b52b140946a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:29.788316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c811e-5627-4e6b-83da-7f58cd54c4d0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:29.801485 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:29.925721 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-050c811e-5627-4e6b-83da-7f58cd54c4d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aafb47ec66f346abbdb27b52b140946a', 'request_id': 'req-050c811e-5627-4e6b-83da-7f58cd54c4d0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:29.931252 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-050c811e-5627-4e6b-83da-7f58cd54c4d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aafb47ec66f346abbdb27b52b140946a', 'request_id': 'req-050c811e-5627-4e6b-83da-7f58cd54c4d0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:29.941855 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-050c811e-5627-4e6b-83da-7f58cd54c4d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aafb47ec66f346abbdb27b52b140946a', 'request_id': 'req-050c811e-5627-4e6b-83da-7f58cd54c4d0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:30.068797 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-050c811e-5627-4e6b-83da-7f58cd54c4d0 admin admin] The token cache is being invalidate because project aafb47ec66f346abbdb27b52b140946a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:46:30.069072 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-050c811e-5627-4e6b-83da-7f58cd54c4d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:30.069670 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-050c811e-5627-4e6b-83da-7f58cd54c4d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:30.071302 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1347/2721] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:46:29 2022] DELETE /identity/v3/projects/aafb47ec66f346abbdb27b52b140946a => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:30.075378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44173c-5c0f-4263-80b0-25e8d733c4d6 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:30.076858 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8c44173c-5c0f-4263-80b0-25e8d733c4d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:30.091391 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44173c-5c0f-4263-80b0-25e8d733c4d6 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:30.148434 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c0706-3624-49d8-87cd-e7bb8d9f360d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:30.149465 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c0706-3624-49d8-87cd-e7bb8d9f360d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:30.149809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c0706-3624-49d8-87cd-e7bb8d9f360d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:30.150183 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c0706-3624-49d8-87cd-e7bb8d9f360d admin admin] PATH_INFO: `/v3/users/03fe64b4adb5415fb453d2c8d85566bd` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:30.151630 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c0706-3624-49d8-87cd-e7bb8d9f360d admin admin] RBAC: Authorizing `identity:delete_user(user_id=03fe64b4adb5415fb453d2c8d85566bd)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:30.154107 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c0706-3624-49d8-87cd-e7bb8d9f360d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:30.397884 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4d3c0706-3624-49d8-87cd-e7bb8d9f360d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03fe64b4adb5415fb453d2c8d85566bd'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:30.398548 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4d3c0706-3624-49d8-87cd-e7bb8d9f360d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03fe64b4adb5415fb453d2c8d85566bd'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:30.406231 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4d3c0706-3624-49d8-87cd-e7bb8d9f360d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03fe64b4adb5415fb453d2c8d85566bd'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:30.406945 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4d3c0706-3624-49d8-87cd-e7bb8d9f360d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03fe64b4adb5415fb453d2c8d85566bd'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:30.480700 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4d3c0706-3624-49d8-87cd-e7bb8d9f360d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03fe64b4adb5415fb453d2c8d85566bd'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:30.500862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1375/2722] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:46:30 2022] DELETE /identity/v3/users/03fe64b4adb5415fb453d2c8d85566bd => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:30.981864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c811e-5627-4e6b-83da-7f58cd54c4d0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:30.981864 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-050c811e-5627-4e6b-83da-7f58cd54c4d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:31.016603 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c811e-5627-4e6b-83da-7f58cd54c4d0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:31.093434 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c1477-94eb-4146-8c25-15493265711b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:31.095767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c1477-94eb-4146-8c25-15493265711b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:31.096245 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c1477-94eb-4146-8c25-15493265711b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:31.097428 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c1477-94eb-4146-8c25-15493265711b admin admin] PATH_INFO: `/v3/projects/add3cd4351ae4bedaf4e48c04f65b2fe` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:31.098967 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c1477-94eb-4146-8c25-15493265711b admin admin] RBAC: Authorizing `identity:delete_project(project_id=add3cd4351ae4bedaf4e48c04f65b2fe)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:31.102164 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c1477-94eb-4146-8c25-15493265711b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:31.275063 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-295c1477-94eb-4146-8c25-15493265711b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'add3cd4351ae4bedaf4e48c04f65b2fe', 'request_id': 'req-295c1477-94eb-4146-8c25-15493265711b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:31.282087 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-295c1477-94eb-4146-8c25-15493265711b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'add3cd4351ae4bedaf4e48c04f65b2fe', 'request_id': 'req-295c1477-94eb-4146-8c25-15493265711b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:31.289931 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-295c1477-94eb-4146-8c25-15493265711b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'add3cd4351ae4bedaf4e48c04f65b2fe', 'request_id': 'req-295c1477-94eb-4146-8c25-15493265711b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:31.360834 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-295c1477-94eb-4146-8c25-15493265711b admin admin] The token cache is being invalidate because project add3cd4351ae4bedaf4e48c04f65b2fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:46:31.361495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-295c1477-94eb-4146-8c25-15493265711b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:31.362483 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-295c1477-94eb-4146-8c25-15493265711b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:46:31.364718 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1348/2723] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:46:30 2022] DELETE /identity/v3/projects/add3cd4351ae4bedaf4e48c04f65b2fe => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:31.371701 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbaea0b-bac8-4b2a-97c8-ef3ee6311a9d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:31.372298 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbaea0b-bac8-4b2a-97c8-ef3ee6311a9d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:31.372298 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbaea0b-bac8-4b2a-97c8-ef3ee6311a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:31.393294 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-bfbaea0b-bac8-4b2a-97c8-ef3ee6311a9d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:31.394637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-bfbaea0b-bac8-4b2a-97c8-ef3ee6311a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:31.409029 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:31.464969 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1376/2724] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:46:31.474799 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32a8ac-a320-4549-83f9-158e6c6e209b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:31.475232 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32a8ac-a320-4549-83f9-158e6c6e209b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:31.475608 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32a8ac-a320-4549-83f9-158e6c6e209b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:31.496251 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-cf32a8ac-a320-4549-83f9-158e6c6e209b None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:31.497521 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cf32a8ac-a320-4549-83f9-158e6c6e209b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:31.509458 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1349/2725] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:46:31.510876 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbaea0b-bac8-4b2a-97c8-ef3ee6311a9d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:31.522314 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbaea0b-bac8-4b2a-97c8-ef3ee6311a9d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:31.527368 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059707d-38b3-4814-a438-5551d79c67dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:31.528417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059707d-38b3-4814-a438-5551d79c67dc admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:31.528780 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059707d-38b3-4814-a438-5551d79c67dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:31.529180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059707d-38b3-4814-a438-5551d79c67dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:31.529824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059707d-38b3-4814-a438-5551d79c67dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:46:31.530127 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059707d-38b3-4814-a438-5551d79c67dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:31.531936 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059707d-38b3-4814-a438-5551d79c67dc admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:31.549336 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1377/2726] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:46:31.550620 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32a8ac-a320-4549-83f9-158e6c6e209b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:31.563361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32a8ac-a320-4549-83f9-158e6c6e209b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:31.570890 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357f551-7984-45ec-b689-22df531d2280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:31.572094 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357f551-7984-45ec-b689-22df531d2280 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:31.572635 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357f551-7984-45ec-b689-22df531d2280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:31.573175 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357f551-7984-45ec-b689-22df531d2280 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:31.576813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357f551-7984-45ec-b689-22df531d2280 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:31.578001 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357f551-7984-45ec-b689-22df531d2280 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:31.704080 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:31.711369 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1350/2727] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:46:31 2022] POST /identity/v3/projects => generated 371 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:46:31.718982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059707d-38b3-4814-a438-5551d79c67dc admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:31.725854 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059707d-38b3-4814-a438-5551d79c67dc admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:31.789076 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3d68e-7fb5-42a7-8ee7-a7e0f50db52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:31.789758 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3d68e-7fb5-42a7-8ee7-a7e0f50db52c admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:31.790192 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3d68e-7fb5-42a7-8ee7-a7e0f50db52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:31.790514 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3d68e-7fb5-42a7-8ee7-a7e0f50db52c admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:31.791234 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3d68e-7fb5-42a7-8ee7-a7e0f50db52c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:31.793411 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3d68e-7fb5-42a7-8ee7-a7e0f50db52c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:31.896093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1378/2728] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:46:31 2022] POST /identity/v3/users => generated 286 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:46:31.899999 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357f551-7984-45ec-b689-22df531d2280 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:31.917822 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357f551-7984-45ec-b689-22df531d2280 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:31.925808 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6c2eb-c684-41eb-b339-abf5ae40fb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:31.926694 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6c2eb-c684-41eb-b339-abf5ae40fb8e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:31.927037 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6c2eb-c684-41eb-b339-abf5ae40fb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:31.927372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6c2eb-c684-41eb-b339-abf5ae40fb8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:31.928266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e6c2eb-c684-41eb-b339-abf5ae40fb8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:31.930728 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e6c2eb-c684-41eb-b339-abf5ae40fb8e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:31.947329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1351/2729] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:46:31.952519 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3d68e-7fb5-42a7-8ee7-a7e0f50db52c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:31.963844 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3d68e-7fb5-42a7-8ee7-a7e0f50db52c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:31.968787 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d5821-6428-461e-a0de-3d2e71cfd17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:31.974584 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d5821-6428-461e-a0de-3d2e71cfd17a admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:31.974584 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d5821-6428-461e-a0de-3d2e71cfd17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:31.974584 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d5821-6428-461e-a0de-3d2e71cfd17a admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:31.975293 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d5821-6428-461e-a0de-3d2e71cfd17a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:31.976914 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d5821-6428-461e-a0de-3d2e71cfd17a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:32.001719 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1379/2730] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:46:32.003217 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6c2eb-c684-41eb-b339-abf5ae40fb8e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:32.015233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6c2eb-c684-41eb-b339-abf5ae40fb8e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:32.021774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a879369-f343-4bfa-8866-bc1bb6ae30d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:32.022861 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a879369-f343-4bfa-8866-bc1bb6ae30d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:32.023155 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a879369-f343-4bfa-8866-bc1bb6ae30d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:32.023429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a879369-f343-4bfa-8866-bc1bb6ae30d2 admin admin] PATH_INFO: `/v3/projects/169e9003449f489f90974df92a8403c7/users/78b8d19b75a74f41802f29f470b8846f/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:32.049427 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a879369-f343-4bfa-8866-bc1bb6ae30d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=169e9003449f489f90974df92a8403c7, user_id=78b8d19b75a74f41802f29f470b8846f, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:32.049427 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a879369-f343-4bfa-8866-bc1bb6ae30d2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:32.191067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1352/2731] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:46:32 2022] PUT /identity/v3/projects/169e9003449f489f90974df92a8403c7/users/78b8d19b75a74f41802f29f470b8846f/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:46:32.203215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0580a2-9cfd-46ba-aa5c-5a3444358a83 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:32.204777 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0580a2-9cfd-46ba-aa5c-5a3444358a83 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:32.205167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0580a2-9cfd-46ba-aa5c-5a3444358a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:32.246269 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-6a0580a2-9cfd-46ba-aa5c-5a3444358a83 None None] MFA Rules not processed for user `78b8d19b75a74f41802f29f470b8846f`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:46:32.254458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6a0580a2-9cfd-46ba-aa5c-5a3444358a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:32.331488 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1380/2732] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:46:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:46:32.344159 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a879369-f343-4bfa-8866-bc1bb6ae30d2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:46:32.347288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0a879369-f343-4bfa-8866-bc1bb6ae30d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:46:32.450103 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a879369-f343-4bfa-8866-bc1bb6ae30d2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:46:32.456217 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67de6e9-48d2-4015-a35c-f738ac2140a3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4d9b5e3826049f2800f1b9424977d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:46:32.457376 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67de6e9-48d2-4015-a35c-f738ac2140a3 service glance] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:46:32.457867 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67de6e9-48d2-4015-a35c-f738ac2140a3 service glance] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:46:32.458340 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67de6e9-48d2-4015-a35c-f738ac2140a3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:46:32.473882 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67de6e9-48d2-4015-a35c-f738ac2140a3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:46:32.476631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67de6e9-48d2-4015-a35c-f738ac2140a3 service glance] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:46:32.494282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1353/2733] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:46:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:46:42.327971 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:46:42.486222 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:47:01.838375 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0580a2-9cfd-46ba-aa5c-5a3444358a83 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:47:01.900783 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0580a2-9cfd-46ba-aa5c-5a3444358a83 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:47:01.905580 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462c617-3dcf-4e5a-afb3-98076228344c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:47:01.906489 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462c617-3dcf-4e5a-afb3-98076228344c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:01.906776 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462c617-3dcf-4e5a-afb3-98076228344c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:01.907053 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462c617-3dcf-4e5a-afb3-98076228344c admin admin] PATH_INFO: `/v3/users/78b8d19b75a74f41802f29f470b8846f` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:01.908254 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462c617-3dcf-4e5a-afb3-98076228344c admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b8d19b75a74f41802f29f470b8846f)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:47:01.910210 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462c617-3dcf-4e5a-afb3-98076228344c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:47:02.110050 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c462c617-3dcf-4e5a-afb3-98076228344c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b8d19b75a74f41802f29f470b8846f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:47:02.110616 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c462c617-3dcf-4e5a-afb3-98076228344c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b8d19b75a74f41802f29f470b8846f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:47:02.117496 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c462c617-3dcf-4e5a-afb3-98076228344c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b8d19b75a74f41802f29f470b8846f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:47:02.118115 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c462c617-3dcf-4e5a-afb3-98076228344c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b8d19b75a74f41802f29f470b8846f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:47:02.164477 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c462c617-3dcf-4e5a-afb3-98076228344c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b8d19b75a74f41802f29f470b8846f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:47:02.183590 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1381/2734] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:47:01 2022] DELETE /identity/v3/users/78b8d19b75a74f41802f29f470b8846f => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:47:02.187921 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2db9a-8cc9-4d54-a847-fdd91cce8e19 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:02.188414 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2db9a-8cc9-4d54-a847-fdd91cce8e19 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:02.188604 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2db9a-8cc9-4d54-a847-fdd91cce8e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:02.206468 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-80b2db9a-8cc9-4d54-a847-fdd91cce8e19 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:47:02.207649 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-80b2db9a-8cc9-4d54-a847-fdd91cce8e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:47:02.265959 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1354/2735] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:47:02.275643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462c617-3dcf-4e5a-afb3-98076228344c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:47:02.277214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c462c617-3dcf-4e5a-afb3-98076228344c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:47:02.334118 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462c617-3dcf-4e5a-afb3-98076228344c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:47:02.337948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7d31e-03fe-4db4-a3d7-b0689fef366c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:47:02.339294 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7d31e-03fe-4db4-a3d7-b0689fef366c service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:02.339294 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7d31e-03fe-4db4-a3d7-b0689fef366c service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:02.339294 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7d31e-03fe-4db4-a3d7-b0689fef366c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:02.348841 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7d31e-03fe-4db4-a3d7-b0689fef366c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:47:02.350322 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7d31e-03fe-4db4-a3d7-b0689fef366c service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:47:02.360274 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1382/2736] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:47:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:47:02.778059 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2db9a-8cc9-4d54-a847-fdd91cce8e19 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:47:02.780052 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-80b2db9a-8cc9-4d54-a847-fdd91cce8e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:47:02.791838 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2db9a-8cc9-4d54-a847-fdd91cce8e19 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:47:02.797221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcea67d-6801-4d91-879a-d01a4995449a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:47:02.798086 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcea67d-6801-4d91-879a-d01a4995449a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:02.798361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcea67d-6801-4d91-879a-d01a4995449a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:02.798628 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcea67d-6801-4d91-879a-d01a4995449a admin admin] PATH_INFO: `/v3/projects/169e9003449f489f90974df92a8403c7` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:02.799808 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcea67d-6801-4d91-879a-d01a4995449a admin admin] RBAC: Authorizing `identity:delete_project(project_id=169e9003449f489f90974df92a8403c7)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:47:02.801784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcea67d-6801-4d91-879a-d01a4995449a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:47:02.913074 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0fcea67d-6801-4d91-879a-d01a4995449a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '169e9003449f489f90974df92a8403c7', 'request_id': 'req-0fcea67d-6801-4d91-879a-d01a4995449a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:47:02.918492 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0fcea67d-6801-4d91-879a-d01a4995449a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '169e9003449f489f90974df92a8403c7', 'request_id': 'req-0fcea67d-6801-4d91-879a-d01a4995449a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:47:02.925255 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0fcea67d-6801-4d91-879a-d01a4995449a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '169e9003449f489f90974df92a8403c7', 'request_id': 'req-0fcea67d-6801-4d91-879a-d01a4995449a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:47:02.998798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0fcea67d-6801-4d91-879a-d01a4995449a admin admin] The token cache is being invalidate because project 169e9003449f489f90974df92a8403c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:47:03.008256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0fcea67d-6801-4d91-879a-d01a4995449a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:47:03.008256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0fcea67d-6801-4d91-879a-d01a4995449a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:47:03.008256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1355/2737] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:47:02 2022] DELETE /identity/v3/projects/169e9003449f489f90974df92a8403c7 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:47:03.008623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574db0e-dc30-4aba-9dfe-3cbd759e3089 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:03.008623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574db0e-dc30-4aba-9dfe-3cbd759e3089 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:03.008623 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574db0e-dc30-4aba-9dfe-3cbd759e3089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:03.025040 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a574db0e-dc30-4aba-9dfe-3cbd759e3089 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:47:03.026389 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a574db0e-dc30-4aba-9dfe-3cbd759e3089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:47:03.043672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:47:03.085672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1383/2738] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:47:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:47:03.147784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62b9f3-e757-45e7-b703-cd058376c121 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:03.148054 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62b9f3-e757-45e7-b703-cd058376c121 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:03.148346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62b9f3-e757-45e7-b703-cd058376c121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:03.167038 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1d62b9f3-e757-45e7-b703-cd058376c121 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:47:03.168163 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1d62b9f3-e757-45e7-b703-cd058376c121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:47:03.176757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1356/2739] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:47:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:47:03.181278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574db0e-dc30-4aba-9dfe-3cbd759e3089 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:47:03.191637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574db0e-dc30-4aba-9dfe-3cbd759e3089 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:47:03.196137 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925c2d2-94c8-4b9e-8335-7bb45b60b178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:47:03.196781 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925c2d2-94c8-4b9e-8335-7bb45b60b178 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:03.197088 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925c2d2-94c8-4b9e-8335-7bb45b60b178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:03.197366 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925c2d2-94c8-4b9e-8335-7bb45b60b178 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:03.198110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925c2d2-94c8-4b9e-8335-7bb45b60b178 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:47:03.198409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925c2d2-94c8-4b9e-8335-7bb45b60b178 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:47:03.199993 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925c2d2-94c8-4b9e-8335-7bb45b60b178 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:47:03.212808 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1384/2740] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:47:03.217833 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62b9f3-e757-45e7-b703-cd058376c121 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:47:03.227196 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62b9f3-e757-45e7-b703-cd058376c121 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:47:03.231977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfadd67-1f47-4492-84c2-3a12dd2b341e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:47:03.232637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfadd67-1f47-4492-84c2-3a12dd2b341e admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:03.232935 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfadd67-1f47-4492-84c2-3a12dd2b341e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:03.233242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfadd67-1f47-4492-84c2-3a12dd2b341e admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:03.234563 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfadd67-1f47-4492-84c2-3a12dd2b341e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:47:03.236476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfadd67-1f47-4492-84c2-3a12dd2b341e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:47:03.344948 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1357/2741] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:47:03 2022] POST /identity/v3/projects => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:47:03.349675 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925c2d2-94c8-4b9e-8335-7bb45b60b178 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:47:03.361998 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925c2d2-94c8-4b9e-8335-7bb45b60b178 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:47:03.421529 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833bf596-cab0-41de-a85e-eec53952c3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:47:03.422373 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bf596-cab0-41de-a85e-eec53952c3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:03.422373 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bf596-cab0-41de-a85e-eec53952c3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:03.422662 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bf596-cab0-41de-a85e-eec53952c3a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:03.423498 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833bf596-cab0-41de-a85e-eec53952c3a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:47:03.425488 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833bf596-cab0-41de-a85e-eec53952c3a8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:47:03.548569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1385/2742] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:47:03 2022] POST /identity/v3/users => generated 291 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:47:03.552528 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfadd67-1f47-4492-84c2-3a12dd2b341e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:47:03.562879 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfadd67-1f47-4492-84c2-3a12dd2b341e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:47:03.567388 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a455e4a-c32e-40c5-a1a1-b2e91cfccc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:47:03.568042 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a455e4a-c32e-40c5-a1a1-b2e91cfccc30 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:03.568393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a455e4a-c32e-40c5-a1a1-b2e91cfccc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:03.568680 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a455e4a-c32e-40c5-a1a1-b2e91cfccc30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:03.569433 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a455e4a-c32e-40c5-a1a1-b2e91cfccc30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:47:03.571026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a455e4a-c32e-40c5-a1a1-b2e91cfccc30 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:47:03.581263 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1358/2743] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:47:03.585185 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833bf596-cab0-41de-a85e-eec53952c3a8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:47:03.595906 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833bf596-cab0-41de-a85e-eec53952c3a8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:47:03.600689 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b808e1b-0909-4a35-b886-65d30d8e629a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:47:03.601386 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b808e1b-0909-4a35-b886-65d30d8e629a admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:03.601672 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b808e1b-0909-4a35-b886-65d30d8e629a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:03.601951 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b808e1b-0909-4a35-b886-65d30d8e629a admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:03.602677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b808e1b-0909-4a35-b886-65d30d8e629a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:47:03.604441 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b808e1b-0909-4a35-b886-65d30d8e629a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:47:03.615093 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1386/2744] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:47:03.619083 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a455e4a-c32e-40c5-a1a1-b2e91cfccc30 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:47:03.629019 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a455e4a-c32e-40c5-a1a1-b2e91cfccc30 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:47:03.633577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab5dbf-a11e-41a7-8d3c-89e16b3f9c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:47:03.634462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab5dbf-a11e-41a7-8d3c-89e16b3f9c3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:03.634743 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab5dbf-a11e-41a7-8d3c-89e16b3f9c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:03.635025 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab5dbf-a11e-41a7-8d3c-89e16b3f9c3a admin admin] PATH_INFO: `/v3/projects/287c3c3374034e3899d7c8b7d970a738/users/03c3d3030c6a4d42a941ae489c5bf125/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:03.649486 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab5dbf-a11e-41a7-8d3c-89e16b3f9c3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=287c3c3374034e3899d7c8b7d970a738, user_id=03c3d3030c6a4d42a941ae489c5bf125, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:47:03.652129 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab5dbf-a11e-41a7-8d3c-89e16b3f9c3a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:47:03.701178 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1359/2745] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:47:03 2022] PUT /identity/v3/projects/287c3c3374034e3899d7c8b7d970a738/users/03c3d3030c6a4d42a941ae489c5bf125/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:47:03.706210 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97840a2e-2eb7-4eae-9b02-51a740f697e5 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:03.706646 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97840a2e-2eb7-4eae-9b02-51a740f697e5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:03.706926 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97840a2e-2eb7-4eae-9b02-51a740f697e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:03.725156 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-97840a2e-2eb7-4eae-9b02-51a740f697e5 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:47:03.726340 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-97840a2e-2eb7-4eae-9b02-51a740f697e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:47:03.784894 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1387/2746] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:47:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:47:03.794305 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab5dbf-a11e-41a7-8d3c-89e16b3f9c3a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:47:03.796018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dbab5dbf-a11e-41a7-8d3c-89e16b3f9c3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:47:03.818137 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:47:03.853018 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab5dbf-a11e-41a7-8d3c-89e16b3f9c3a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:47:03.857351 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6709c-a043-4bd5-8098-f2d3fe800621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:47:03.857999 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6709c-a043-4bd5-8098-f2d3fe800621 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:03.858285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6709c-a043-4bd5-8098-f2d3fe800621 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:03.858577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6709c-a043-4bd5-8098-f2d3fe800621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:03.868568 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6709c-a043-4bd5-8098-f2d3fe800621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:47:03.870033 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6709c-a043-4bd5-8098-f2d3fe800621 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:47:03.880283 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1360/2747] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:47:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:47:05.171142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b3625-d030-4c87-ba61-d5e796c38d2d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:05.171690 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b3625-d030-4c87-ba61-d5e796c38d2d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:05.171891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b3625-d030-4c87-ba61-d5e796c38d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:05.202747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-f56b3625-d030-4c87-ba61-d5e796c38d2d None None] MFA Rules not processed for user `03c3d3030c6a4d42a941ae489c5bf125`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:47:05.203986 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f56b3625-d030-4c87-ba61-d5e796c38d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:47:05.221299 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:47:05.269809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1388/2748] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:47:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:47:05.281645 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6709c-a043-4bd5-8098-f2d3fe800621 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:47:05.283163 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c1d6709c-a043-4bd5-8098-f2d3fe800621 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:47:05.341859 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6709c-a043-4bd5-8098-f2d3fe800621 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:47:05.345380 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9f7bd-3bb8-4e40-a349-2297a94fd3ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:47:05.346037 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9f7bd-3bb8-4e40-a349-2297a94fd3ff service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:05.346301 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9f7bd-3bb8-4e40-a349-2297a94fd3ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:05.346573 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9f7bd-3bb8-4e40-a349-2297a94fd3ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:05.355165 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9f7bd-3bb8-4e40-a349-2297a94fd3ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:47:05.356623 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9f7bd-3bb8-4e40-a349-2297a94fd3ff service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:47:05.365632 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1361/2749] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:47:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:47:07.000966 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfe5d1-866a-4316-bafc-c45b90dd1fd9 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:07.001562 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfe5d1-866a-4316-bafc-c45b90dd1fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:07.001679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfe5d1-866a-4316-bafc-c45b90dd1fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:07.019684 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-37cfe5d1-866a-4316-bafc-c45b90dd1fd9 None None] MFA Rules not processed for user `03c3d3030c6a4d42a941ae489c5bf125`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:47:07.021142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-37cfe5d1-866a-4316-bafc-c45b90dd1fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:47:07.028428 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1389/2750] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:47:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:47:15.361505 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=43 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:47:17.014003 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=22 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:47:24.063785 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9f7bd-3bb8-4e40-a349-2297a94fd3ff service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:47:24.075250 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9f7bd-3bb8-4e40-a349-2297a94fd3ff service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:47:24.080390 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ab81a-7582-4b2e-9007-8453cb401254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:47:24.081439 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ab81a-7582-4b2e-9007-8453cb401254 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:24.083797 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ab81a-7582-4b2e-9007-8453cb401254 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:24.084500 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ab81a-7582-4b2e-9007-8453cb401254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:24.086935 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-554ab81a-7582-4b2e-9007-8453cb401254 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:47:24.099645 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554ab81a-7582-4b2e-9007-8453cb401254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:47:24.102250 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554ab81a-7582-4b2e-9007-8453cb401254 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:47:24.113440 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1362/2751] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:47:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:47:24.269535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cfe5d1-866a-4316-bafc-c45b90dd1fd9 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:47:24.292609 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cfe5d1-866a-4316-bafc-c45b90dd1fd9 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:47:24.298137 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30103e9b-5e2c-41d5-b514-ca109bd42433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:47:24.299940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30103e9b-5e2c-41d5-b514-ca109bd42433 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:24.300341 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30103e9b-5e2c-41d5-b514-ca109bd42433 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:24.300888 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30103e9b-5e2c-41d5-b514-ca109bd42433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:24.305610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-30103e9b-5e2c-41d5-b514-ca109bd42433 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:47:24.408072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30103e9b-5e2c-41d5-b514-ca109bd42433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:47:24.409589 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30103e9b-5e2c-41d5-b514-ca109bd42433 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:47:24.422692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1390/2752] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:47:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:47:34.109265 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:47:34.416344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=34 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:47:38.497849 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538377b-c59c-4d0f-ad5d-194366ead34e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:47:38.498393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538377b-c59c-4d0f-ad5d-194366ead34e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:47:38.498393 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538377b-c59c-4d0f-ad5d-194366ead34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:47:38.517382 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7538377b-c59c-4d0f-ad5d-194366ead34e None None] MFA Rules not processed for user `52ddf59fe764404eb1c6ee745e21d352`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:47:38.518605 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7538377b-c59c-4d0f-ad5d-194366ead34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:47:38.575117 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1363/2753] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:47:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:47:48.568104 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=25 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:48:21.882270 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052edbe3-e44d-4e4b-b978-7cdb6fdc7472 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:48:21.882806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052edbe3-e44d-4e4b-b978-7cdb6fdc7472 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:48:21.882806 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052edbe3-e44d-4e4b-b978-7cdb6fdc7472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:48:21.900087 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-052edbe3-e44d-4e4b-b978-7cdb6fdc7472 None None] MFA Rules not processed for user `03c3d3030c6a4d42a941ae489c5bf125`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:48:21.901323 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-052edbe3-e44d-4e4b-b978-7cdb6fdc7472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:48:21.908524 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1391/2754] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:48:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:48:21.948694 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538377b-c59c-4d0f-ad5d-194366ead34e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:48:21.950021 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7538377b-c59c-4d0f-ad5d-194366ead34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:48:22.010306 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538377b-c59c-4d0f-ad5d-194366ead34e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:48:22.014047 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa73fd4-ee33-435c-abb3-21fc41dfe462 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4d9b5e3826049f2800f1b9424977d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:48:22.014699 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa73fd4-ee33-435c-abb3-21fc41dfe462 service glance] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:48:22.014983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa73fd4-ee33-435c-abb3-21fc41dfe462 service glance] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:48:22.015262 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa73fd4-ee33-435c-abb3-21fc41dfe462 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:48:22.017214 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3fa73fd4-ee33-435c-abb3-21fc41dfe462 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:48:22.031410 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa73fd4-ee33-435c-abb3-21fc41dfe462 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:48:22.032840 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa73fd4-ee33-435c-abb3-21fc41dfe462 service glance] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:48:22.044615 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1364/2755] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:48:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:48:22.638269 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052edbe3-e44d-4e4b-b978-7cdb6fdc7472 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:48:22.639651 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-052edbe3-e44d-4e4b-b978-7cdb6fdc7472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:48:22.647540 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa73fd4-ee33-435c-abb3-21fc41dfe462 service glance] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:48:22.658098 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa73fd4-ee33-435c-abb3-21fc41dfe462 service glance] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:48:22.662878 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034330a-fb11-4b51-acd3-7daf81926099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:48:22.664783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034330a-fb11-4b51-acd3-7daf81926099 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:48:22.664783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034330a-fb11-4b51-acd3-7daf81926099 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:48:22.664783 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034330a-fb11-4b51-acd3-7daf81926099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:48:22.665938 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0034330a-fb11-4b51-acd3-7daf81926099 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:48:22.679947 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0034330a-fb11-4b51-acd3-7daf81926099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:48:22.681443 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0034330a-fb11-4b51-acd3-7daf81926099 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:48:22.693360 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1365/2756] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:48:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:48:22.714924 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052edbe3-e44d-4e4b-b978-7cdb6fdc7472 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:48:22.719821 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63077d7-c679-4ee0-b8a8-7ab7ce9f0cb5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc0d7bd7e4547f4a8d90ffc2c2e1d03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:48:22.720470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63077d7-c679-4ee0-b8a8-7ab7ce9f0cb5 service placement] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:48:22.720753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63077d7-c679-4ee0-b8a8-7ab7ce9f0cb5 service placement] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:48:22.721111 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63077d7-c679-4ee0-b8a8-7ab7ce9f0cb5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:48:22.723491 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c63077d7-c679-4ee0-b8a8-7ab7ce9f0cb5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:48:22.744152 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63077d7-c679-4ee0-b8a8-7ab7ce9f0cb5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:48:22.744152 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63077d7-c679-4ee0-b8a8-7ab7ce9f0cb5 service placement] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:48:22.747737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1392/2757] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:48:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:48:30.923916 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034330a-fb11-4b51-acd3-7daf81926099 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:48:30.925466 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0034330a-fb11-4b51-acd3-7daf81926099 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:48:30.936551 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034330a-fb11-4b51-acd3-7daf81926099 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:48:30.941275 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af1fd1-871e-4b9d-b639-01adc7cd2d2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:48:30.941275 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af1fd1-871e-4b9d-b639-01adc7cd2d2f service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:48:30.941275 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af1fd1-871e-4b9d-b639-01adc7cd2d2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:48:30.941275 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af1fd1-871e-4b9d-b639-01adc7cd2d2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:48:30.942945 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-26af1fd1-871e-4b9d-b639-01adc7cd2d2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:48:30.953288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af1fd1-871e-4b9d-b639-01adc7cd2d2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:48:30.958501 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af1fd1-871e-4b9d-b639-01adc7cd2d2f service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:48:30.962673 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1366/2758] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:48:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:48:32.743313 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=35 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:48:40.958934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=38 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:48:56.569702 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63077d7-c679-4ee0-b8a8-7ab7ce9f0cb5 service placement] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:48:56.572578 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c63077d7-c679-4ee0-b8a8-7ab7ce9f0cb5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:48:56.591891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63077d7-c679-4ee0-b8a8-7ab7ce9f0cb5 service placement] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:48:56.597910 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ca6df-09fa-4c00-9218-3ad4b9e85df0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:48:56.598607 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ca6df-09fa-4c00-9218-3ad4b9e85df0 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:48:56.598949 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ca6df-09fa-4c00-9218-3ad4b9e85df0 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:48:56.599402 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ca6df-09fa-4c00-9218-3ad4b9e85df0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:48:56.601532 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-df5ca6df-09fa-4c00-9218-3ad4b9e85df0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:48:56.615175 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ca6df-09fa-4c00-9218-3ad4b9e85df0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:48:56.616888 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ca6df-09fa-4c00-9218-3ad4b9e85df0 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:48:56.628666 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1393/2759] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:48:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:49:02.854024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf32d1-cf77-42c6-8e5d-f6f998422ca1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:49:02.854586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf32d1-cf77-42c6-8e5d-f6f998422ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:49:02.854586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf32d1-cf77-42c6-8e5d-f6f998422ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:49:02.872587 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-54bf32d1-cf77-42c6-8e5d-f6f998422ca1 None None] MFA Rules not processed for user `52ddf59fe764404eb1c6ee745e21d352`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:49:02.882587 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-54bf32d1-cf77-42c6-8e5d-f6f998422ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:49:02.882587 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1367/2760] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:49:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:49:06.623541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=6 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:49:07.900581 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ca6df-09fa-4c00-9218-3ad4b9e85df0 service nova] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:49:07.913020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ca6df-09fa-4c00-9218-3ad4b9e85df0 service nova] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:49:07.918758 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311ff57-6008-4484-9cd7-022c1e8687e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:49:07.919428 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311ff57-6008-4484-9cd7-022c1e8687e0 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:49:07.919713 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311ff57-6008-4484-9cd7-022c1e8687e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:49:07.919998 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311ff57-6008-4484-9cd7-022c1e8687e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:49:07.921837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8311ff57-6008-4484-9cd7-022c1e8687e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:49:07.985785 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311ff57-6008-4484-9cd7-022c1e8687e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:49:07.987541 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311ff57-6008-4484-9cd7-022c1e8687e0 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:49:07.998919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1394/2761] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:49:07 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:49:11.975525 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf32d1-cf77-42c6-8e5d-f6f998422ca1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:49:11.985340 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf32d1-cf77-42c6-8e5d-f6f998422ca1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:49:11.988877 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c652e-42a8-4c98-a100-627508a1ad4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:49:11.989542 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c652e-42a8-4c98-a100-627508a1ad4b service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:49:11.989825 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c652e-42a8-4c98-a100-627508a1ad4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:49:11.990099 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c652e-42a8-4c98-a100-627508a1ad4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:49:11.991907 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2e4c652e-42a8-4c98-a100-627508a1ad4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:49:12.004028 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c652e-42a8-4c98-a100-627508a1ad4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:49:12.005495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c652e-42a8-4c98-a100-627508a1ad4b service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:49:12.015665 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1368/2762] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:49:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:49:17.992888 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=28 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:49:20.072800 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311ff57-6008-4484-9cd7-022c1e8687e0 service nova] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:49:20.075858 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8311ff57-6008-4484-9cd7-022c1e8687e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:49:20.091567 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311ff57-6008-4484-9cd7-022c1e8687e0 service nova] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:49:20.096697 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0859c-d6fe-4724-b5dc-ed46b69d3d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:49:20.097802 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0859c-d6fe-4724-b5dc-ed46b69d3d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:49:20.098085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0859c-d6fe-4724-b5dc-ed46b69d3d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:49:20.098438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0859c-d6fe-4724-b5dc-ed46b69d3d4a admin admin] PATH_INFO: `/v3/users/346ec431db3947ea97f1f2e3b27db0a0` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:49:20.099994 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0859c-d6fe-4724-b5dc-ed46b69d3d4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=346ec431db3947ea97f1f2e3b27db0a0)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:49:20.101991 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0859c-d6fe-4724-b5dc-ed46b69d3d4a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:49:20.325827 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-3dd0859c-d6fe-4724-b5dc-ed46b69d3d4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '346ec431db3947ea97f1f2e3b27db0a0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:20.326506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-3dd0859c-d6fe-4724-b5dc-ed46b69d3d4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '346ec431db3947ea97f1f2e3b27db0a0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:20.333556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-3dd0859c-d6fe-4724-b5dc-ed46b69d3d4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '346ec431db3947ea97f1f2e3b27db0a0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:20.334184 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-3dd0859c-d6fe-4724-b5dc-ed46b69d3d4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '346ec431db3947ea97f1f2e3b27db0a0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:20.419306 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-3dd0859c-d6fe-4724-b5dc-ed46b69d3d4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '346ec431db3947ea97f1f2e3b27db0a0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:20.438373 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1395/2763] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:49:20 2022] DELETE /identity/v3/users/346ec431db3947ea97f1f2e3b27db0a0 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:49:20.657463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c652e-42a8-4c98-a100-627508a1ad4b service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:49:20.669155 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c652e-42a8-4c98-a100-627508a1ad4b service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:49:20.742965 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617015c5-1b6b-4eeb-8d75-70dcc8b4f8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:49:20.743726 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617015c5-1b6b-4eeb-8d75-70dcc8b4f8f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:49:20.744006 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617015c5-1b6b-4eeb-8d75-70dcc8b4f8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:49:20.744279 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617015c5-1b6b-4eeb-8d75-70dcc8b4f8f6 admin admin] PATH_INFO: `/v3/projects/bfc85651f1d947e3b52f02b4cf8e26b3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:49:20.745510 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617015c5-1b6b-4eeb-8d75-70dcc8b4f8f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfc85651f1d947e3b52f02b4cf8e26b3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:49:20.747456 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617015c5-1b6b-4eeb-8d75-70dcc8b4f8f6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:49:20.874701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-617015c5-1b6b-4eeb-8d75-70dcc8b4f8f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfc85651f1d947e3b52f02b4cf8e26b3', 'request_id': 'req-617015c5-1b6b-4eeb-8d75-70dcc8b4f8f6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:20.881073 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-617015c5-1b6b-4eeb-8d75-70dcc8b4f8f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfc85651f1d947e3b52f02b4cf8e26b3', 'request_id': 'req-617015c5-1b6b-4eeb-8d75-70dcc8b4f8f6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:20.888613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-617015c5-1b6b-4eeb-8d75-70dcc8b4f8f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfc85651f1d947e3b52f02b4cf8e26b3', 'request_id': 'req-617015c5-1b6b-4eeb-8d75-70dcc8b4f8f6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:20.975277 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-617015c5-1b6b-4eeb-8d75-70dcc8b4f8f6 admin admin] The token cache is being invalidate because project bfc85651f1d947e3b52f02b4cf8e26b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:49:20.975734 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-617015c5-1b6b-4eeb-8d75-70dcc8b4f8f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:20.976535 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-617015c5-1b6b-4eeb-8d75-70dcc8b4f8f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:20.978286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1369/2764] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:49:20 2022] DELETE /identity/v3/projects/bfc85651f1d947e3b52f02b4cf8e26b3 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:49:20.982554 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0859c-d6fe-4724-b5dc-ed46b69d3d4a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:49:20.984773 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3dd0859c-d6fe-4724-b5dc-ed46b69d3d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:49:20.998745 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0859c-d6fe-4724-b5dc-ed46b69d3d4a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:49:21.066161 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4cf74-89e9-471f-b10e-58253f30f65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:49:21.067217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4cf74-89e9-471f-b10e-58253f30f65d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:49:21.067561 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4cf74-89e9-471f-b10e-58253f30f65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:49:21.067845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4cf74-89e9-471f-b10e-58253f30f65d admin admin] PATH_INFO: `/v3/users/c71fa913b3944742b4a9f89272668294` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:49:21.069175 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4cf74-89e9-471f-b10e-58253f30f65d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c71fa913b3944742b4a9f89272668294)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:49:21.071105 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4cf74-89e9-471f-b10e-58253f30f65d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:49:21.234549 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7ca4cf74-89e9-471f-b10e-58253f30f65d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c71fa913b3944742b4a9f89272668294'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:21.235686 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7ca4cf74-89e9-471f-b10e-58253f30f65d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c71fa913b3944742b4a9f89272668294'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:21.251579 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7ca4cf74-89e9-471f-b10e-58253f30f65d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c71fa913b3944742b4a9f89272668294'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:21.252627 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7ca4cf74-89e9-471f-b10e-58253f30f65d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c71fa913b3944742b4a9f89272668294'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:21.314464 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7ca4cf74-89e9-471f-b10e-58253f30f65d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c71fa913b3944742b4a9f89272668294'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:21.330546 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:49:21.346424 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1396/2765] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:49:20 2022] DELETE /identity/v3/users/c71fa913b3944742b4a9f89272668294 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:49:21.615119 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617015c5-1b6b-4eeb-8d75-70dcc8b4f8f6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:49:21.616655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-617015c5-1b6b-4eeb-8d75-70dcc8b4f8f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:49:21.633839 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617015c5-1b6b-4eeb-8d75-70dcc8b4f8f6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:49:21.703170 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b3bb9-9c40-48cb-a5e6-817bceb649d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:49:21.704368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b3bb9-9c40-48cb-a5e6-817bceb649d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:49:21.704777 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b3bb9-9c40-48cb-a5e6-817bceb649d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:49:21.705270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b3bb9-9c40-48cb-a5e6-817bceb649d9 admin admin] PATH_INFO: `/v3/projects/fd6e957a48f940f4a64e4e525ffb9bff` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:49:21.706960 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b3bb9-9c40-48cb-a5e6-817bceb649d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd6e957a48f940f4a64e4e525ffb9bff)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:49:21.709698 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b3bb9-9c40-48cb-a5e6-817bceb649d9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:49:21.724090 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:49:21.817988 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d76b3bb9-9c40-48cb-a5e6-817bceb649d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd6e957a48f940f4a64e4e525ffb9bff', 'request_id': 'req-d76b3bb9-9c40-48cb-a5e6-817bceb649d9'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:21.824518 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d76b3bb9-9c40-48cb-a5e6-817bceb649d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd6e957a48f940f4a64e4e525ffb9bff', 'request_id': 'req-d76b3bb9-9c40-48cb-a5e6-817bceb649d9'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:21.831147 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d76b3bb9-9c40-48cb-a5e6-817bceb649d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd6e957a48f940f4a64e4e525ffb9bff', 'request_id': 'req-d76b3bb9-9c40-48cb-a5e6-817bceb649d9'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:21.887709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d76b3bb9-9c40-48cb-a5e6-817bceb649d9 admin admin] The token cache is being invalidate because project fd6e957a48f940f4a64e4e525ffb9bff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:49:21.888866 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d76b3bb9-9c40-48cb-a5e6-817bceb649d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:21.889711 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d76b3bb9-9c40-48cb-a5e6-817bceb649d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:21.891309 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1370/2766] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:49:21 2022] DELETE /identity/v3/projects/fd6e957a48f940f4a64e4e525ffb9bff => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:49:28.162731 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4cf74-89e9-471f-b10e-58253f30f65d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:49:28.164231 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7ca4cf74-89e9-471f-b10e-58253f30f65d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:49:28.240320 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4cf74-89e9-471f-b10e-58253f30f65d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:49:28.303144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d712b9-c9cd-4a4e-9ac3-c76f4aa9bac5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc0d7bd7e4547f4a8d90ffc2c2e1d03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:49:28.303919 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d712b9-c9cd-4a4e-9ac3-c76f4aa9bac5 service placement] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:49:28.304213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d712b9-c9cd-4a4e-9ac3-c76f4aa9bac5 service placement] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:49:28.304506 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d712b9-c9cd-4a4e-9ac3-c76f4aa9bac5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:49:28.306466 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a5d712b9-c9cd-4a4e-9ac3-c76f4aa9bac5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:49:28.318431 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b3bb9-9c40-48cb-a5e6-817bceb649d9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:49:28.318431 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d76b3bb9-9c40-48cb-a5e6-817bceb649d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:49:28.320229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d712b9-c9cd-4a4e-9ac3-c76f4aa9bac5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:49:28.322024 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d712b9-c9cd-4a4e-9ac3-c76f4aa9bac5 service placement] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:49:28.334173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1397/2767] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:49:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:49:28.394733 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b3bb9-9c40-48cb-a5e6-817bceb649d9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:49:28.399452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f88849-479a-4998-86ab-5cfd4032c72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:49:28.400328 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f88849-479a-4998-86ab-5cfd4032c72f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:49:28.400900 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f88849-479a-4998-86ab-5cfd4032c72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:49:28.401203 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f88849-479a-4998-86ab-5cfd4032c72f admin admin] PATH_INFO: `/v3/users/03c3d3030c6a4d42a941ae489c5bf125` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:49:28.402399 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f88849-479a-4998-86ab-5cfd4032c72f admin admin] RBAC: Authorizing `identity:delete_user(user_id=03c3d3030c6a4d42a941ae489c5bf125)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:49:28.404406 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f88849-479a-4998-86ab-5cfd4032c72f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:49:28.576819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b9f88849-479a-4998-86ab-5cfd4032c72f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03c3d3030c6a4d42a941ae489c5bf125'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:28.577539 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b9f88849-479a-4998-86ab-5cfd4032c72f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03c3d3030c6a4d42a941ae489c5bf125'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:28.584859 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b9f88849-479a-4998-86ab-5cfd4032c72f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03c3d3030c6a4d42a941ae489c5bf125'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:28.585549 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b9f88849-479a-4998-86ab-5cfd4032c72f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03c3d3030c6a4d42a941ae489c5bf125'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:28.597551 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b9f88849-479a-4998-86ab-5cfd4032c72f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03c3d3030c6a4d42a941ae489c5bf125'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:28.675596 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1371/2768] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:49:28 2022] DELETE /identity/v3/users/03c3d3030c6a4d42a941ae489c5bf125 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:49:28.956843 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d712b9-c9cd-4a4e-9ac3-c76f4aa9bac5 service placement] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:49:28.969888 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d712b9-c9cd-4a4e-9ac3-c76f4aa9bac5 service placement] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:49:29.028972 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b7490-df1b-4bd8-b169-cbf8e4f83beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:49:29.029850 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b7490-df1b-4bd8-b169-cbf8e4f83beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:49:29.030129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b7490-df1b-4bd8-b169-cbf8e4f83beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:49:29.030398 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b7490-df1b-4bd8-b169-cbf8e4f83beb admin admin] PATH_INFO: `/v3/projects/287c3c3374034e3899d7c8b7d970a738` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:49:29.031562 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b7490-df1b-4bd8-b169-cbf8e4f83beb admin admin] RBAC: Authorizing `identity:delete_project(project_id=287c3c3374034e3899d7c8b7d970a738)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:49:29.033534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b7490-df1b-4bd8-b169-cbf8e4f83beb admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:49:29.134853 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:49:29.153112 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-3e9b7490-df1b-4bd8-b169-cbf8e4f83beb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '287c3c3374034e3899d7c8b7d970a738', 'request_id': 'req-3e9b7490-df1b-4bd8-b169-cbf8e4f83beb'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:29.159470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-3e9b7490-df1b-4bd8-b169-cbf8e4f83beb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '287c3c3374034e3899d7c8b7d970a738', 'request_id': 'req-3e9b7490-df1b-4bd8-b169-cbf8e4f83beb'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:29.213144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-3e9b7490-df1b-4bd8-b169-cbf8e4f83beb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '287c3c3374034e3899d7c8b7d970a738', 'request_id': 'req-3e9b7490-df1b-4bd8-b169-cbf8e4f83beb'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:29.224552 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-3e9b7490-df1b-4bd8-b169-cbf8e4f83beb admin admin] The token cache is being invalidate because project 287c3c3374034e3899d7c8b7d970a738 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:49:29.225846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-3e9b7490-df1b-4bd8-b169-cbf8e4f83beb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:29.225846 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-3e9b7490-df1b-4bd8-b169-cbf8e4f83beb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:49:29.227350 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1398/2769] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:49:28 2022] DELETE /identity/v3/projects/287c3c3374034e3899d7c8b7d970a738 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:49:38.671933 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=72,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:49:39.221549 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:19.769556 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f88849-479a-4998-86ab-5cfd4032c72f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:19.771000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b9f88849-479a-4998-86ab-5cfd4032c72f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:19.855587 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f88849-479a-4998-86ab-5cfd4032c72f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:19.904485 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c28d7-9d43-465d-bc03-a37ca91f622a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:19.905177 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c28d7-9d43-465d-bc03-a37ca91f622a service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:19.905459 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c28d7-9d43-465d-bc03-a37ca91f622a service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:19.905733 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c28d7-9d43-465d-bc03-a37ca91f622a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:19.908067 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e51c28d7-9d43-465d-bc03-a37ca91f622a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:19.985740 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c28d7-9d43-465d-bc03-a37ca91f622a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:19.987168 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c28d7-9d43-465d-bc03-a37ca91f622a service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:19.996116 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1372/2770] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:50:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:50:20.092475 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b7490-df1b-4bd8-b169-cbf8e4f83beb admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:20.093907 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3e9b7490-df1b-4bd8-b169-cbf8e4f83beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:20.177064 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b7490-df1b-4bd8-b169-cbf8e4f83beb admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:20.180666 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1321b2-a6ca-46bf-bfdb-0c1ba21596a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:20.181362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1321b2-a6ca-46bf-bfdb-0c1ba21596a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:20.181639 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1321b2-a6ca-46bf-bfdb-0c1ba21596a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:20.181911 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1321b2-a6ca-46bf-bfdb-0c1ba21596a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:20.183685 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1c1321b2-a6ca-46bf-bfdb-0c1ba21596a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:20.273643 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1321b2-a6ca-46bf-bfdb-0c1ba21596a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:20.275347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1321b2-a6ca-46bf-bfdb-0c1ba21596a6 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:20.294796 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1399/2771] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:50:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:50:23.738288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c28d7-9d43-465d-bc03-a37ca91f622a service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:23.739784 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e51c28d7-9d43-465d-bc03-a37ca91f622a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:23.773832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:23.812000 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c28d7-9d43-465d-bc03-a37ca91f622a service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:23.817778 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7cf94-1daf-40c5-9341-08fe68da9ecf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc0d7bd7e4547f4a8d90ffc2c2e1d03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:23.818439 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7cf94-1daf-40c5-9341-08fe68da9ecf service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:23.818727 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7cf94-1daf-40c5-9341-08fe68da9ecf service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:23.819002 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7cf94-1daf-40c5-9341-08fe68da9ecf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:23.820820 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e9a7cf94-1daf-40c5-9341-08fe68da9ecf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:23.833411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7cf94-1daf-40c5-9341-08fe68da9ecf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:23.834998 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7cf94-1daf-40c5-9341-08fe68da9ecf service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:23.847902 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1373/2772] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:50:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:50:23.856798 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1321b2-a6ca-46bf-bfdb-0c1ba21596a6 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:23.858308 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1c1321b2-a6ca-46bf-bfdb-0c1ba21596a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:23.872756 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1321b2-a6ca-46bf-bfdb-0c1ba21596a6 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:23.879187 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c7c80-22a9-40c4-98d0-fee4794baf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:23.879714 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c7c80-22a9-40c4-98d0-fee4794baf71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:23.880063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c7c80-22a9-40c4-98d0-fee4794baf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:23.880427 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c7c80-22a9-40c4-98d0-fee4794baf71 admin admin] PATH_INFO: `/v3/users/52ddf59fe764404eb1c6ee745e21d352` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:23.881891 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c7c80-22a9-40c4-98d0-fee4794baf71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52ddf59fe764404eb1c6ee745e21d352)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:23.884074 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c7c80-22a9-40c4-98d0-fee4794baf71 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:24.041816 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:24.043654 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-345c7c80-22a9-40c4-98d0-fee4794baf71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52ddf59fe764404eb1c6ee745e21d352'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:24.044311 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-345c7c80-22a9-40c4-98d0-fee4794baf71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52ddf59fe764404eb1c6ee745e21d352'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:24.051303 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-345c7c80-22a9-40c4-98d0-fee4794baf71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52ddf59fe764404eb1c6ee745e21d352'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:24.052008 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-345c7c80-22a9-40c4-98d0-fee4794baf71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52ddf59fe764404eb1c6ee745e21d352'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:24.119758 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-345c7c80-22a9-40c4-98d0-fee4794baf71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52ddf59fe764404eb1c6ee745e21d352'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:24.137939 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1400/2773] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:50:23 2022] DELETE /identity/v3/users/52ddf59fe764404eb1c6ee745e21d352 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:24.373054 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7cf94-1daf-40c5-9341-08fe68da9ecf service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:24.382697 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7cf94-1daf-40c5-9341-08fe68da9ecf service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:24.443497 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c960f45-605e-4684-a559-3b1403bbf748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:24.444368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c960f45-605e-4684-a559-3b1403bbf748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:24.444650 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c960f45-605e-4684-a559-3b1403bbf748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:24.444979 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c960f45-605e-4684-a559-3b1403bbf748 admin admin] PATH_INFO: `/v3/projects/0899cad1b7c64a0199b3466d4c7cfefb` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:24.446194 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c960f45-605e-4684-a559-3b1403bbf748 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0899cad1b7c64a0199b3466d4c7cfefb)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:24.448202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c960f45-605e-4684-a559-3b1403bbf748 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:24.531623 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3c960f45-605e-4684-a559-3b1403bbf748 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0899cad1b7c64a0199b3466d4c7cfefb', 'request_id': 'req-3c960f45-605e-4684-a559-3b1403bbf748'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:24.537738 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3c960f45-605e-4684-a559-3b1403bbf748 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0899cad1b7c64a0199b3466d4c7cfefb', 'request_id': 'req-3c960f45-605e-4684-a559-3b1403bbf748'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:24.543925 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3c960f45-605e-4684-a559-3b1403bbf748 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0899cad1b7c64a0199b3466d4c7cfefb', 'request_id': 'req-3c960f45-605e-4684-a559-3b1403bbf748'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:24.594949 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3c960f45-605e-4684-a559-3b1403bbf748 admin admin] The token cache is being invalidate because project 0899cad1b7c64a0199b3466d4c7cfefb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:50:24.595272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3c960f45-605e-4684-a559-3b1403bbf748 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:24.595915 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3c960f45-605e-4684-a559-3b1403bbf748 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:24.597635 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1374/2774] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:50:24 2022] DELETE /identity/v3/projects/0899cad1b7c64a0199b3466d4c7cfefb => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:24.602537 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d4f50-8ede-4518-9d8a-e94b59c223d5 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:24.602864 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d4f50-8ede-4518-9d8a-e94b59c223d5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:24.603147 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d4f50-8ede-4518-9d8a-e94b59c223d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:24.649747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9e3d4f50-8ede-4518-9d8a-e94b59c223d5 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:24.651219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9e3d4f50-8ede-4518-9d8a-e94b59c223d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:24.708789 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1401/2775] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:24.717360 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0387e0-2d1c-4928-b13d-9ceca0ede04c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:24.717671 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0387e0-2d1c-4928-b13d-9ceca0ede04c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:24.717942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0387e0-2d1c-4928-b13d-9ceca0ede04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:24.736570 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0f0387e0-2d1c-4928-b13d-9ceca0ede04c None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:24.737811 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0f0387e0-2d1c-4928-b13d-9ceca0ede04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:24.745702 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1375/2776] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:24.750155 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d4f50-8ede-4518-9d8a-e94b59c223d5 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:24.760557 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d4f50-8ede-4518-9d8a-e94b59c223d5 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:24.765227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b69f81-9553-4736-ab4e-f2bcbc686187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:24.765876 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b69f81-9553-4736-ab4e-f2bcbc686187 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:24.766162 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b69f81-9553-4736-ab4e-f2bcbc686187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:24.766433 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b69f81-9553-4736-ab4e-f2bcbc686187 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:24.767154 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b69f81-9553-4736-ab4e-f2bcbc686187 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:50:24.767461 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b69f81-9553-4736-ab4e-f2bcbc686187 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:24.769077 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b69f81-9553-4736-ab4e-f2bcbc686187 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:24.780698 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1402/2777] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:50:24.785109 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0387e0-2d1c-4928-b13d-9ceca0ede04c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:24.794581 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0387e0-2d1c-4928-b13d-9ceca0ede04c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:24.800226 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca843d-5026-43ed-810c-425558123276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:24.800885 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca843d-5026-43ed-810c-425558123276 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:24.801192 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca843d-5026-43ed-810c-425558123276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:24.801468 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca843d-5026-43ed-810c-425558123276 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:24.802163 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca843d-5026-43ed-810c-425558123276 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:24.804045 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca843d-5026-43ed-810c-425558123276 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:24.876973 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1376/2778] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:50:24 2022] POST /identity/v3/projects => generated 365 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:50:24.881350 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b69f81-9553-4736-ab4e-f2bcbc686187 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:24.894680 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b69f81-9553-4736-ab4e-f2bcbc686187 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:24.951464 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1a8ae-2e76-4627-afc2-997d091f292e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:24.952259 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1a8ae-2e76-4627-afc2-997d091f292e admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:24.952544 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1a8ae-2e76-4627-afc2-997d091f292e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:24.952902 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1a8ae-2e76-4627-afc2-997d091f292e admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:24.953656 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1a8ae-2e76-4627-afc2-997d091f292e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:24.956095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1a8ae-2e76-4627-afc2-997d091f292e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:25.034728 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1403/2779] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:50:24 2022] POST /identity/v3/users => generated 283 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:50:25.039134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca843d-5026-43ed-810c-425558123276 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:25.049793 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca843d-5026-43ed-810c-425558123276 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:25.054473 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7178fbb-7d7c-4a02-84b3-8e6fee0465ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:25.055143 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7178fbb-7d7c-4a02-84b3-8e6fee0465ff admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:25.055429 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7178fbb-7d7c-4a02-84b3-8e6fee0465ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:25.055708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7178fbb-7d7c-4a02-84b3-8e6fee0465ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:25.056608 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7178fbb-7d7c-4a02-84b3-8e6fee0465ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:25.058195 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7178fbb-7d7c-4a02-84b3-8e6fee0465ff admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:25.069952 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1377/2780] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:50:25.074475 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1a8ae-2e76-4627-afc2-997d091f292e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:25.085678 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1a8ae-2e76-4627-afc2-997d091f292e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:25.090705 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c88087-efd3-43b3-a6e1-6bc77cac236b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:25.091386 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c88087-efd3-43b3-a6e1-6bc77cac236b admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:25.091705 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c88087-efd3-43b3-a6e1-6bc77cac236b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:25.091984 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c88087-efd3-43b3-a6e1-6bc77cac236b admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:25.092718 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c88087-efd3-43b3-a6e1-6bc77cac236b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:25.094489 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c88087-efd3-43b3-a6e1-6bc77cac236b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:25.105085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1404/2781] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:50:25.110346 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7178fbb-7d7c-4a02-84b3-8e6fee0465ff admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:25.126632 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7178fbb-7d7c-4a02-84b3-8e6fee0465ff admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:25.133096 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee71f0-e2d2-4cce-8dfc-eca8d4164c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:25.133995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee71f0-e2d2-4cce-8dfc-eca8d4164c6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:25.134274 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee71f0-e2d2-4cce-8dfc-eca8d4164c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:25.134582 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee71f0-e2d2-4cce-8dfc-eca8d4164c6e admin admin] PATH_INFO: `/v3/projects/65a5704fced04dae95f1fa7baa2972e7/users/a38b3c7a57f74d8f9b35efca4098deb1/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:25.138029 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:25.150573 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee71f0-e2d2-4cce-8dfc-eca8d4164c6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=65a5704fced04dae95f1fa7baa2972e7, user_id=a38b3c7a57f74d8f9b35efca4098deb1, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:25.153286 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee71f0-e2d2-4cce-8dfc-eca8d4164c6e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:25.241209 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1378/2782] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:50:25 2022] PUT /identity/v3/projects/65a5704fced04dae95f1fa7baa2972e7/users/a38b3c7a57f74d8f9b35efca4098deb1/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:25.251940 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2283bf-db36-43c6-9d9e-7d79f30de939 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:25.252321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2283bf-db36-43c6-9d9e-7d79f30de939 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:25.252634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2283bf-db36-43c6-9d9e-7d79f30de939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:25.265578 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:25.281207 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ba2283bf-db36-43c6-9d9e-7d79f30de939 None None] MFA Rules not processed for user `a38b3c7a57f74d8f9b35efca4098deb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:25.282962 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ba2283bf-db36-43c6-9d9e-7d79f30de939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:25.359458 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1405/2783] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:25.364259 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee71f0-e2d2-4cce-8dfc-eca8d4164c6e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:25.428629 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee71f0-e2d2-4cce-8dfc-eca8d4164c6e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:25.438850 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de18c8-406e-4009-9407-c91706c1bcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:25.439523 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de18c8-406e-4009-9407-c91706c1bcbb admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:25.439807 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de18c8-406e-4009-9407-c91706c1bcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:25.440087 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de18c8-406e-4009-9407-c91706c1bcbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:25.440794 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de18c8-406e-4009-9407-c91706c1bcbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:25.442754 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de18c8-406e-4009-9407-c91706c1bcbb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:25.534994 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1379/2784] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:50:25 2022] POST /identity/v3/projects => generated 369 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:50:25.538988 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2283bf-db36-43c6-9d9e-7d79f30de939 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:25.550932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2283bf-db36-43c6-9d9e-7d79f30de939 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:25.612154 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc490603-686d-4eac-a5e8-ee68296d2247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:25.612740 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc490603-686d-4eac-a5e8-ee68296d2247 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:25.613042 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc490603-686d-4eac-a5e8-ee68296d2247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:25.613321 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc490603-686d-4eac-a5e8-ee68296d2247 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:25.614023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc490603-686d-4eac-a5e8-ee68296d2247 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:25.615950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc490603-686d-4eac-a5e8-ee68296d2247 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:25.686977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1406/2785] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:50:25 2022] POST /identity/v3/users => generated 291 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:50:25.691548 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de18c8-406e-4009-9407-c91706c1bcbb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:25.701970 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de18c8-406e-4009-9407-c91706c1bcbb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:25.706875 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91985aa3-239e-40a2-b193-bd8eabc7c682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:25.707597 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91985aa3-239e-40a2-b193-bd8eabc7c682 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:25.707905 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91985aa3-239e-40a2-b193-bd8eabc7c682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:25.708180 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91985aa3-239e-40a2-b193-bd8eabc7c682 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:25.708902 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91985aa3-239e-40a2-b193-bd8eabc7c682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:25.710626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91985aa3-239e-40a2-b193-bd8eabc7c682 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:25.722235 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1380/2786] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:50:25.728817 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc490603-686d-4eac-a5e8-ee68296d2247 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:25.740500 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc490603-686d-4eac-a5e8-ee68296d2247 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:25.745547 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a15e42-2bcf-4f11-b0ad-ecfdbda001af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:25.746383 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a15e42-2bcf-4f11-b0ad-ecfdbda001af admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:25.746667 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a15e42-2bcf-4f11-b0ad-ecfdbda001af admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:25.746944 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a15e42-2bcf-4f11-b0ad-ecfdbda001af admin admin] PATH_INFO: `/v3/projects/5079f51b3e9a46d2b9b954c97957f4ac/users/2eb1174e1f664007a92ddf7e8a3a8d7c/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:25.763270 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a15e42-2bcf-4f11-b0ad-ecfdbda001af admin admin] RBAC: Authorizing `identity:create_grant(project_id=5079f51b3e9a46d2b9b954c97957f4ac, user_id=2eb1174e1f664007a92ddf7e8a3a8d7c, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:25.766042 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a15e42-2bcf-4f11-b0ad-ecfdbda001af admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:25.829550 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1407/2787] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:50:25 2022] PUT /identity/v3/projects/5079f51b3e9a46d2b9b954c97957f4ac/users/2eb1174e1f664007a92ddf7e8a3a8d7c/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:25.833721 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91985aa3-239e-40a2-b193-bd8eabc7c682 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:25.843942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91985aa3-239e-40a2-b193-bd8eabc7c682 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:25.901495 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ccd67-a507-4ac5-a0d2-f7b78fe1bbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:25.902078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ccd67-a507-4ac5-a0d2-f7b78fe1bbc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:25.902364 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ccd67-a507-4ac5-a0d2-f7b78fe1bbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:25.902641 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ccd67-a507-4ac5-a0d2-f7b78fe1bbc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:25.903372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ccd67-a507-4ac5-a0d2-f7b78fe1bbc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:25.904985 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ccd67-a507-4ac5-a0d2-f7b78fe1bbc8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:25.915733 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1381/2788] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:50:25.919774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a15e42-2bcf-4f11-b0ad-ecfdbda001af admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:25.931384 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a15e42-2bcf-4f11-b0ad-ecfdbda001af admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:25.936249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6a4e5-e29b-403f-bb08-11f74a862df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:25.937178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6a4e5-e29b-403f-bb08-11f74a862df2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:25.937462 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6a4e5-e29b-403f-bb08-11f74a862df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:25.937737 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6a4e5-e29b-403f-bb08-11f74a862df2 admin admin] PATH_INFO: `/v3/projects/5079f51b3e9a46d2b9b954c97957f4ac/users/2eb1174e1f664007a92ddf7e8a3a8d7c/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:25.940219 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6a4e5-e29b-403f-bb08-11f74a862df2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5079f51b3e9a46d2b9b954c97957f4ac, user_id=2eb1174e1f664007a92ddf7e8a3a8d7c, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:25.942936 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6a4e5-e29b-403f-bb08-11f74a862df2 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:25.953523 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1408/2789] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:50:25 2022] PUT /identity/v3/projects/5079f51b3e9a46d2b9b954c97957f4ac/users/2eb1174e1f664007a92ddf7e8a3a8d7c/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:25.965833 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501a43e-78a1-406a-ba5f-a4aa43badf3d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:25.966121 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501a43e-78a1-406a-ba5f-a4aa43badf3d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:25.966401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501a43e-78a1-406a-ba5f-a4aa43badf3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:25.996437 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a501a43e-78a1-406a-ba5f-a4aa43badf3d None None] MFA Rules not processed for user `2eb1174e1f664007a92ddf7e8a3a8d7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:25.997702 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a501a43e-78a1-406a-ba5f-a4aa43badf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:26.049644 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:26.076545 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1382/2790] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:25 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:26.088453 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6a4e5-e29b-403f-bb08-11f74a862df2 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:26.090425 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d1f6a4e5-e29b-403f-bb08-11f74a862df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:26.145591 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:26.149601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6a4e5-e29b-403f-bb08-11f74a862df2 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:26.155322 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec547d6-d8b6-4f71-b29e-1fa8af95f158 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:26.155322 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec547d6-d8b6-4f71-b29e-1fa8af95f158 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:26.155322 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec547d6-d8b6-4f71-b29e-1fa8af95f158 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:26.155322 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec547d6-d8b6-4f71-b29e-1fa8af95f158 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:26.165056 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec547d6-d8b6-4f71-b29e-1fa8af95f158 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:26.167043 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec547d6-d8b6-4f71-b29e-1fa8af95f158 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:26.178108 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1409/2791] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:50:26 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:50:26.276482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501a43e-78a1-406a-ba5f-a4aa43badf3d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:26.278069 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a501a43e-78a1-406a-ba5f-a4aa43badf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:26.291320 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501a43e-78a1-406a-ba5f-a4aa43badf3d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:26.295750 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21424cf9-5cbe-4e07-a3c7-4bf6f17cc115 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:26.296394 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21424cf9-5cbe-4e07-a3c7-4bf6f17cc115 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:26.296676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21424cf9-5cbe-4e07-a3c7-4bf6f17cc115 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:26.296953 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21424cf9-5cbe-4e07-a3c7-4bf6f17cc115 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:26.356313 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21424cf9-5cbe-4e07-a3c7-4bf6f17cc115 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:26.357788 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21424cf9-5cbe-4e07-a3c7-4bf6f17cc115 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:26.370850 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1383/2792] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:50:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:50:30.821166 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec547d6-d8b6-4f71-b29e-1fa8af95f158 service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:30.883098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec547d6-d8b6-4f71-b29e-1fa8af95f158 service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:30.887630 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c56f3-69fe-46ed-8a3a-b0c0185f05dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:30.888494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c56f3-69fe-46ed-8a3a-b0c0185f05dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:30.888769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c56f3-69fe-46ed-8a3a-b0c0185f05dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:30.889065 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c56f3-69fe-46ed-8a3a-b0c0185f05dc admin admin] PATH_INFO: `/v3/users/a38b3c7a57f74d8f9b35efca4098deb1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:30.890237 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c56f3-69fe-46ed-8a3a-b0c0185f05dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a38b3c7a57f74d8f9b35efca4098deb1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:30.892150 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c56f3-69fe-46ed-8a3a-b0c0185f05dc admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:31.100398 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ef5c56f3-69fe-46ed-8a3a-b0c0185f05dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a38b3c7a57f74d8f9b35efca4098deb1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:31.100981 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ef5c56f3-69fe-46ed-8a3a-b0c0185f05dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a38b3c7a57f74d8f9b35efca4098deb1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:31.108008 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ef5c56f3-69fe-46ed-8a3a-b0c0185f05dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a38b3c7a57f74d8f9b35efca4098deb1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:31.108647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ef5c56f3-69fe-46ed-8a3a-b0c0185f05dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a38b3c7a57f74d8f9b35efca4098deb1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:31.176576 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ef5c56f3-69fe-46ed-8a3a-b0c0185f05dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a38b3c7a57f74d8f9b35efca4098deb1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:31.196242 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1410/2793] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:50:30 2022] DELETE /identity/v3/users/a38b3c7a57f74d8f9b35efca4098deb1 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:31.201052 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466ea4a-6870-4710-a26d-bf5ecca1c65c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:31.201792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466ea4a-6870-4710-a26d-bf5ecca1c65c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:31.201792 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466ea4a-6870-4710-a26d-bf5ecca1c65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:31.220280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8466ea4a-6870-4710-a26d-bf5ecca1c65c None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:31.221542 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8466ea4a-6870-4710-a26d-bf5ecca1c65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:31.286785 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1384/2794] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:31.296908 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c56f3-69fe-46ed-8a3a-b0c0185f05dc admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:31.298386 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ef5c56f3-69fe-46ed-8a3a-b0c0185f05dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:31.355614 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c56f3-69fe-46ed-8a3a-b0c0185f05dc admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:31.359208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922ad19-25ca-4430-b441-5b22f7d9d4f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:31.359845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922ad19-25ca-4430-b441-5b22f7d9d4f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:31.360118 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922ad19-25ca-4430-b441-5b22f7d9d4f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:31.360388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922ad19-25ca-4430-b441-5b22f7d9d4f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:31.369802 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922ad19-25ca-4430-b441-5b22f7d9d4f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:31.371247 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922ad19-25ca-4430-b441-5b22f7d9d4f3 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:31.381504 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1411/2795] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:50:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:50:31.796185 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466ea4a-6870-4710-a26d-bf5ecca1c65c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:31.797710 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8466ea4a-6870-4710-a26d-bf5ecca1c65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:31.811257 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466ea4a-6870-4710-a26d-bf5ecca1c65c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:31.815959 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d119174-39ba-4cde-ba84-bb67f8592ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:31.816913 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d119174-39ba-4cde-ba84-bb67f8592ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:31.817233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d119174-39ba-4cde-ba84-bb67f8592ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:31.817655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d119174-39ba-4cde-ba84-bb67f8592ad5 admin admin] PATH_INFO: `/v3/projects/65a5704fced04dae95f1fa7baa2972e7` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:31.818874 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d119174-39ba-4cde-ba84-bb67f8592ad5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65a5704fced04dae95f1fa7baa2972e7)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:31.820863 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d119174-39ba-4cde-ba84-bb67f8592ad5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:31.910220 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9d119174-39ba-4cde-ba84-bb67f8592ad5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65a5704fced04dae95f1fa7baa2972e7', 'request_id': 'req-9d119174-39ba-4cde-ba84-bb67f8592ad5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:31.916545 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9d119174-39ba-4cde-ba84-bb67f8592ad5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65a5704fced04dae95f1fa7baa2972e7', 'request_id': 'req-9d119174-39ba-4cde-ba84-bb67f8592ad5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:31.923988 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9d119174-39ba-4cde-ba84-bb67f8592ad5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65a5704fced04dae95f1fa7baa2972e7', 'request_id': 'req-9d119174-39ba-4cde-ba84-bb67f8592ad5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:31.988527 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9d119174-39ba-4cde-ba84-bb67f8592ad5 admin admin] The token cache is being invalidate because project 65a5704fced04dae95f1fa7baa2972e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:50:31.988813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9d119174-39ba-4cde-ba84-bb67f8592ad5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:31.989487 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9d119174-39ba-4cde-ba84-bb67f8592ad5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:31.991112 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1385/2796] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:50:31 2022] DELETE /identity/v3/projects/65a5704fced04dae95f1fa7baa2972e7 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:31.995259 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922ad19-25ca-4430-b441-5b22f7d9d4f3 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:32.005537 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922ad19-25ca-4430-b441-5b22f7d9d4f3 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:32.014416 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:32.062722 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ef693-15f4-4f15-bfb5-c4ada0c4ebed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:32.063636 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ef693-15f4-4f15-bfb5-c4ada0c4ebed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:32.063912 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ef693-15f4-4f15-bfb5-c4ada0c4ebed admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:32.064687 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ef693-15f4-4f15-bfb5-c4ada0c4ebed admin admin] PATH_INFO: `/v3/users/2eb1174e1f664007a92ddf7e8a3a8d7c` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:32.065894 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ef693-15f4-4f15-bfb5-c4ada0c4ebed admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eb1174e1f664007a92ddf7e8a3a8d7c)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:32.067824 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ef693-15f4-4f15-bfb5-c4ada0c4ebed admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:32.192281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f95ef693-15f4-4f15-bfb5-c4ada0c4ebed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eb1174e1f664007a92ddf7e8a3a8d7c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:32.192953 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f95ef693-15f4-4f15-bfb5-c4ada0c4ebed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eb1174e1f664007a92ddf7e8a3a8d7c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:32.200429 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f95ef693-15f4-4f15-bfb5-c4ada0c4ebed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eb1174e1f664007a92ddf7e8a3a8d7c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:32.201197 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f95ef693-15f4-4f15-bfb5-c4ada0c4ebed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eb1174e1f664007a92ddf7e8a3a8d7c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:32.262973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f95ef693-15f4-4f15-bfb5-c4ada0c4ebed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eb1174e1f664007a92ddf7e8a3a8d7c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:32.281319 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1412/2797] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:50:31 2022] DELETE /identity/v3/users/2eb1174e1f664007a92ddf7e8a3a8d7c => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:50:32.686547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d119174-39ba-4cde-ba84-bb67f8592ad5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:32.687992 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9d119174-39ba-4cde-ba84-bb67f8592ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:32.696586 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:32.701736 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d119174-39ba-4cde-ba84-bb67f8592ad5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:32.759765 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be519ade-2a31-4f08-b1cd-bc3d9e0d99e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:32.760631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be519ade-2a31-4f08-b1cd-bc3d9e0d99e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:32.760913 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be519ade-2a31-4f08-b1cd-bc3d9e0d99e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:32.761219 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be519ade-2a31-4f08-b1cd-bc3d9e0d99e0 admin admin] PATH_INFO: `/v3/projects/5079f51b3e9a46d2b9b954c97957f4ac` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:32.762388 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be519ade-2a31-4f08-b1cd-bc3d9e0d99e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5079f51b3e9a46d2b9b954c97957f4ac)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:32.764365 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be519ade-2a31-4f08-b1cd-bc3d9e0d99e0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:32.869916 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-be519ade-2a31-4f08-b1cd-bc3d9e0d99e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5079f51b3e9a46d2b9b954c97957f4ac', 'request_id': 'req-be519ade-2a31-4f08-b1cd-bc3d9e0d99e0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:32.876223 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-be519ade-2a31-4f08-b1cd-bc3d9e0d99e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5079f51b3e9a46d2b9b954c97957f4ac', 'request_id': 'req-be519ade-2a31-4f08-b1cd-bc3d9e0d99e0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:32.884200 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-be519ade-2a31-4f08-b1cd-bc3d9e0d99e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5079f51b3e9a46d2b9b954c97957f4ac', 'request_id': 'req-be519ade-2a31-4f08-b1cd-bc3d9e0d99e0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:32.955662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-be519ade-2a31-4f08-b1cd-bc3d9e0d99e0 admin admin] The token cache is being invalidate because project 5079f51b3e9a46d2b9b954c97957f4ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:50:32.956030 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-be519ade-2a31-4f08-b1cd-bc3d9e0d99e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:32.956748 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-be519ade-2a31-4f08-b1cd-bc3d9e0d99e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:32.958454 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1386/2798] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:50:32 2022] DELETE /identity/v3/projects/5079f51b3e9a46d2b9b954c97957f4ac => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:32.965192 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258b430-9ae6-4995-af0e-4f2fef1318f6 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:32.965192 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258b430-9ae6-4995-af0e-4f2fef1318f6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:32.965192 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258b430-9ae6-4995-af0e-4f2fef1318f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:32.982915 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-0258b430-9ae6-4995-af0e-4f2fef1318f6 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:32.984149 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0258b430-9ae6-4995-af0e-4f2fef1318f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:33.042252 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1413/2799] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:33.050532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb6f04-dc11-41b1-8221-d20655791019 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:33.050854 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb6f04-dc11-41b1-8221-d20655791019 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:33.051135 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb6f04-dc11-41b1-8221-d20655791019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:33.070246 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3bbb6f04-dc11-41b1-8221-d20655791019 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:33.071487 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3bbb6f04-dc11-41b1-8221-d20655791019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:33.080266 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1387/2800] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:33.084490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258b430-9ae6-4995-af0e-4f2fef1318f6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:33.094102 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258b430-9ae6-4995-af0e-4f2fef1318f6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:33.099248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aad541-3e8a-4daa-a400-896872b3d8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:33.099900 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aad541-3e8a-4daa-a400-896872b3d8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:33.100178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aad541-3e8a-4daa-a400-896872b3d8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:33.100453 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aad541-3e8a-4daa-a400-896872b3d8b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:33.101206 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aad541-3e8a-4daa-a400-896872b3d8b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:50:33.101514 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aad541-3e8a-4daa-a400-896872b3d8b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:33.103116 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aad541-3e8a-4daa-a400-896872b3d8b6 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:33.114939 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1414/2801] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:50:33.119395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb6f04-dc11-41b1-8221-d20655791019 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:33.128840 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb6f04-dc11-41b1-8221-d20655791019 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:33.133802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711d894-a149-4be1-8c3d-83f2f200cec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:33.134461 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711d894-a149-4be1-8c3d-83f2f200cec0 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:33.134738 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711d894-a149-4be1-8c3d-83f2f200cec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:33.135035 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711d894-a149-4be1-8c3d-83f2f200cec0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:33.135802 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711d894-a149-4be1-8c3d-83f2f200cec0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:33.137781 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711d894-a149-4be1-8c3d-83f2f200cec0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:33.220469 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1388/2802] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:50:33 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 08:50:33.224755 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aad541-3e8a-4daa-a400-896872b3d8b6 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:33.235239 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aad541-3e8a-4daa-a400-896872b3d8b6 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:33.253454 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:33.292633 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf466825-bd35-4090-9181-bfaa4f09b043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:33.293334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf466825-bd35-4090-9181-bfaa4f09b043 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:33.293622 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf466825-bd35-4090-9181-bfaa4f09b043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:33.293897 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf466825-bd35-4090-9181-bfaa4f09b043 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:33.294583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf466825-bd35-4090-9181-bfaa4f09b043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:33.296565 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf466825-bd35-4090-9181-bfaa4f09b043 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:33.389490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1415/2803] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:50:33 2022] POST /identity/v3/users => generated 294 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:50:33.394441 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711d894-a149-4be1-8c3d-83f2f200cec0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:33.405296 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711d894-a149-4be1-8c3d-83f2f200cec0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:33.410753 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620c49b-5af8-4f6d-9d8d-1dd9b5d7879f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:33.411418 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620c49b-5af8-4f6d-9d8d-1dd9b5d7879f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:33.411695 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620c49b-5af8-4f6d-9d8d-1dd9b5d7879f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:33.411966 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620c49b-5af8-4f6d-9d8d-1dd9b5d7879f admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:33.412713 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620c49b-5af8-4f6d-9d8d-1dd9b5d7879f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:33.414368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620c49b-5af8-4f6d-9d8d-1dd9b5d7879f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:33.425627 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1389/2804] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:50:33.430105 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf466825-bd35-4090-9181-bfaa4f09b043 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:33.440383 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf466825-bd35-4090-9181-bfaa4f09b043 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:33.446044 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351cdc5-2a84-4a85-9f26-c0342cb26e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:33.446727 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351cdc5-2a84-4a85-9f26-c0342cb26e7c admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:33.447078 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351cdc5-2a84-4a85-9f26-c0342cb26e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:33.447362 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351cdc5-2a84-4a85-9f26-c0342cb26e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:33.448100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351cdc5-2a84-4a85-9f26-c0342cb26e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:33.449864 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351cdc5-2a84-4a85-9f26-c0342cb26e7c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:33.460178 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1416/2805] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:50:33.464630 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620c49b-5af8-4f6d-9d8d-1dd9b5d7879f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:33.476384 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620c49b-5af8-4f6d-9d8d-1dd9b5d7879f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:33.488814 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879425a8-577f-4620-b150-721520069b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:33.489755 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879425a8-577f-4620-b150-721520069b84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:33.490043 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879425a8-577f-4620-b150-721520069b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:33.490333 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879425a8-577f-4620-b150-721520069b84 admin admin] PATH_INFO: `/v3/projects/f7639587db51499dbd12d13867ed4034/users/431680c7cf43477e87c67022ee5569cd/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:33.507112 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879425a8-577f-4620-b150-721520069b84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7639587db51499dbd12d13867ed4034, user_id=431680c7cf43477e87c67022ee5569cd, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:33.509873 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879425a8-577f-4620-b150-721520069b84 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:33.576977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1390/2806] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:50:33 2022] PUT /identity/v3/projects/f7639587db51499dbd12d13867ed4034/users/431680c7cf43477e87c67022ee5569cd/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:33.587692 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a871b-561e-4099-9ff4-a2a497d0fed3 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:33.587977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a871b-561e-4099-9ff4-a2a497d0fed3 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:33.588259 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a871b-561e-4099-9ff4-a2a497d0fed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:33.617228 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ce8a871b-561e-4099-9ff4-a2a497d0fed3 None None] MFA Rules not processed for user `431680c7cf43477e87c67022ee5569cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:33.618437 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ce8a871b-561e-4099-9ff4-a2a497d0fed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:33.683152 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1417/2807] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:33.688397 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879425a8-577f-4620-b150-721520069b84 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:33.741664 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:33.792984 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879425a8-577f-4620-b150-721520069b84 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:33.801209 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae139624-1c2c-4e2e-b6df-4e1113f7c026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:33.802331 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae139624-1c2c-4e2e-b6df-4e1113f7c026 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:33.802835 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae139624-1c2c-4e2e-b6df-4e1113f7c026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:33.803336 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae139624-1c2c-4e2e-b6df-4e1113f7c026 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:33.804629 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae139624-1c2c-4e2e-b6df-4e1113f7c026 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:33.808412 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae139624-1c2c-4e2e-b6df-4e1113f7c026 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:33.904987 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1391/2808] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:50:33 2022] POST /identity/v3/projects => generated 387 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:50:33.909833 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a871b-561e-4099-9ff4-a2a497d0fed3 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:33.921055 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a871b-561e-4099-9ff4-a2a497d0fed3 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:33.978156 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfe088-c50b-492d-8676-e368aff8545c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:33.978822 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfe088-c50b-492d-8676-e368aff8545c admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:33.979103 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfe088-c50b-492d-8676-e368aff8545c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:33.979379 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfe088-c50b-492d-8676-e368aff8545c admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:33.980063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acfe088-c50b-492d-8676-e368aff8545c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:33.982037 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acfe088-c50b-492d-8676-e368aff8545c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:34.045726 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1418/2809] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:50:33 2022] POST /identity/v3/users => generated 300 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:50:34.050114 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae139624-1c2c-4e2e-b6df-4e1113f7c026 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:34.060372 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae139624-1c2c-4e2e-b6df-4e1113f7c026 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:34.065148 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fa222-3216-4ede-bd9d-ca9b5dded949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:34.065810 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fa222-3216-4ede-bd9d-ca9b5dded949 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:34.066094 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fa222-3216-4ede-bd9d-ca9b5dded949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:34.066373 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fa222-3216-4ede-bd9d-ca9b5dded949 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:34.067093 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1fa222-3216-4ede-bd9d-ca9b5dded949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:34.068777 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1fa222-3216-4ede-bd9d-ca9b5dded949 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:34.079219 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1392/2810] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:50:34.083180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfe088-c50b-492d-8676-e368aff8545c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:34.093028 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfe088-c50b-492d-8676-e368aff8545c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:34.098031 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbc2ef-cf77-4921-be63-b1db324ab5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:34.098896 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbc2ef-cf77-4921-be63-b1db324ab5b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:34.099176 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbc2ef-cf77-4921-be63-b1db324ab5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:34.099447 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbc2ef-cf77-4921-be63-b1db324ab5b5 admin admin] PATH_INFO: `/v3/projects/a9fe3ebcd02e46f181e59f8423f25e7f/users/27db225222374a3ba16d7f1f15fdbf54/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:34.114822 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bbc2ef-cf77-4921-be63-b1db324ab5b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9fe3ebcd02e46f181e59f8423f25e7f, user_id=27db225222374a3ba16d7f1f15fdbf54, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:34.117539 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bbc2ef-cf77-4921-be63-b1db324ab5b5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:34.119755 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:34.211795 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1419/2811] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:50:34 2022] PUT /identity/v3/projects/a9fe3ebcd02e46f181e59f8423f25e7f/users/27db225222374a3ba16d7f1f15fdbf54/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:34.215876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fa222-3216-4ede-bd9d-ca9b5dded949 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:34.226665 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fa222-3216-4ede-bd9d-ca9b5dded949 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:34.284582 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ad9f5-dd68-421e-8ca5-cf2a5cf04e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:34.285289 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ad9f5-dd68-421e-8ca5-cf2a5cf04e95 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:34.285569 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ad9f5-dd68-421e-8ca5-cf2a5cf04e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:34.285847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ad9f5-dd68-421e-8ca5-cf2a5cf04e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:34.286577 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ad9f5-dd68-421e-8ca5-cf2a5cf04e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:34.288170 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ad9f5-dd68-421e-8ca5-cf2a5cf04e95 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:34.299100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1393/2812] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:50:34.303200 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbc2ef-cf77-4921-be63-b1db324ab5b5 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:34.313528 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbc2ef-cf77-4921-be63-b1db324ab5b5 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:34.318691 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b29c3-1cd4-4665-a74e-dda438b5dd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:34.319569 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b29c3-1cd4-4665-a74e-dda438b5dd24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:34.319870 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b29c3-1cd4-4665-a74e-dda438b5dd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:34.320144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b29c3-1cd4-4665-a74e-dda438b5dd24 admin admin] PATH_INFO: `/v3/projects/a9fe3ebcd02e46f181e59f8423f25e7f/users/27db225222374a3ba16d7f1f15fdbf54/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:34.322188 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b29c3-1cd4-4665-a74e-dda438b5dd24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9fe3ebcd02e46f181e59f8423f25e7f, user_id=27db225222374a3ba16d7f1f15fdbf54, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:34.325099 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b29c3-1cd4-4665-a74e-dda438b5dd24 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:34.336311 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1420/2813] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:50:34 2022] PUT /identity/v3/projects/a9fe3ebcd02e46f181e59f8423f25e7f/users/27db225222374a3ba16d7f1f15fdbf54/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:34.401237 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8a206-965e-4828-beb2-1ed3bd6ebac4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:34.401532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8a206-965e-4828-beb2-1ed3bd6ebac4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:34.401819 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8a206-965e-4828-beb2-1ed3bd6ebac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:34.430242 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9cb8a206-965e-4828-beb2-1ed3bd6ebac4 None None] MFA Rules not processed for user `27db225222374a3ba16d7f1f15fdbf54`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:34.431463 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9cb8a206-965e-4828-beb2-1ed3bd6ebac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:34.500610 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1394/2814] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:34.512801 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b29c3-1cd4-4665-a74e-dda438b5dd24 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:34.514659 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-103b29c3-1cd4-4665-a74e-dda438b5dd24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:34.574677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b29c3-1cd4-4665-a74e-dda438b5dd24 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:34.578200 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f390e-f166-446e-9a51-183a977c5ce0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:34.578836 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f390e-f166-446e-9a51-183a977c5ce0 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:34.579117 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f390e-f166-446e-9a51-183a977c5ce0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:34.579390 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f390e-f166-446e-9a51-183a977c5ce0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:34.588791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f390e-f166-446e-9a51-183a977c5ce0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:34.590272 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f390e-f166-446e-9a51-183a977c5ce0 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:34.600072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1421/2815] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:50:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:50:34.628442 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8a206-965e-4828-beb2-1ed3bd6ebac4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:34.654562 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:34.690449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8a206-965e-4828-beb2-1ed3bd6ebac4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:34.695216 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84805a75-37bb-4ebe-9568-c6c5f3294f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:34.696119 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84805a75-37bb-4ebe-9568-c6c5f3294f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:34.696401 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84805a75-37bb-4ebe-9568-c6c5f3294f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:34.696756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84805a75-37bb-4ebe-9568-c6c5f3294f0f admin admin] PATH_INFO: `/v3/users/431680c7cf43477e87c67022ee5569cd` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:34.698022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84805a75-37bb-4ebe-9568-c6c5f3294f0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=431680c7cf43477e87c67022ee5569cd)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:34.699941 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84805a75-37bb-4ebe-9568-c6c5f3294f0f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:34.847069 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-84805a75-37bb-4ebe-9568-c6c5f3294f0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '431680c7cf43477e87c67022ee5569cd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:34.847717 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-84805a75-37bb-4ebe-9568-c6c5f3294f0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '431680c7cf43477e87c67022ee5569cd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:34.854983 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-84805a75-37bb-4ebe-9568-c6c5f3294f0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '431680c7cf43477e87c67022ee5569cd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:34.855660 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-84805a75-37bb-4ebe-9568-c6c5f3294f0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '431680c7cf43477e87c67022ee5569cd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:34.868581 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-84805a75-37bb-4ebe-9568-c6c5f3294f0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '431680c7cf43477e87c67022ee5569cd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:34.966496 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1395/2816] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:50:34 2022] DELETE /identity/v3/users/431680c7cf43477e87c67022ee5569cd => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:34.971085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29a0ad-36c5-44ed-b290-622563f4ed35 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:34.971371 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29a0ad-36c5-44ed-b290-622563f4ed35 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:34.971657 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29a0ad-36c5-44ed-b290-622563f4ed35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:34.989179 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-df29a0ad-36c5-44ed-b290-622563f4ed35 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:34.990385 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-df29a0ad-36c5-44ed-b290-622563f4ed35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:35.052963 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1422/2817] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:35.063011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84805a75-37bb-4ebe-9568-c6c5f3294f0f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:35.065108 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-84805a75-37bb-4ebe-9568-c6c5f3294f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:35.123742 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84805a75-37bb-4ebe-9568-c6c5f3294f0f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:35.128152 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e4897-10eb-4c5f-9409-4af967a1033a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:35.128800 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e4897-10eb-4c5f-9409-4af967a1033a service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:35.129117 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e4897-10eb-4c5f-9409-4af967a1033a service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:35.129397 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e4897-10eb-4c5f-9409-4af967a1033a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:35.139369 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e4897-10eb-4c5f-9409-4af967a1033a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:35.140807 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e4897-10eb-4c5f-9409-4af967a1033a service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:35.152153 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1396/2818] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:50:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:50:35.544823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29a0ad-36c5-44ed-b290-622563f4ed35 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:35.546378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-df29a0ad-36c5-44ed-b290-622563f4ed35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:35.560106 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29a0ad-36c5-44ed-b290-622563f4ed35 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:35.565812 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edba4e2-0814-4165-9072-d8f7e3fc4808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:35.566707 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edba4e2-0814-4165-9072-d8f7e3fc4808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:35.566979 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edba4e2-0814-4165-9072-d8f7e3fc4808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:35.567278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edba4e2-0814-4165-9072-d8f7e3fc4808 admin admin] PATH_INFO: `/v3/projects/f7639587db51499dbd12d13867ed4034` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:35.569188 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edba4e2-0814-4165-9072-d8f7e3fc4808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7639587db51499dbd12d13867ed4034)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:35.571081 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edba4e2-0814-4165-9072-d8f7e3fc4808 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:35.694605 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5edba4e2-0814-4165-9072-d8f7e3fc4808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7639587db51499dbd12d13867ed4034', 'request_id': 'req-5edba4e2-0814-4165-9072-d8f7e3fc4808'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:35.701016 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5edba4e2-0814-4165-9072-d8f7e3fc4808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7639587db51499dbd12d13867ed4034', 'request_id': 'req-5edba4e2-0814-4165-9072-d8f7e3fc4808'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:35.743012 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5edba4e2-0814-4165-9072-d8f7e3fc4808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7639587db51499dbd12d13867ed4034', 'request_id': 'req-5edba4e2-0814-4165-9072-d8f7e3fc4808'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:35.754825 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5edba4e2-0814-4165-9072-d8f7e3fc4808 admin admin] The token cache is being invalidate because project f7639587db51499dbd12d13867ed4034 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:50:35.755110 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5edba4e2-0814-4165-9072-d8f7e3fc4808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:35.755792 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5edba4e2-0814-4165-9072-d8f7e3fc4808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:35.757375 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1423/2819] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:50:35 2022] DELETE /identity/v3/projects/f7639587db51499dbd12d13867ed4034 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:35.761547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e4897-10eb-4c5f-9409-4af967a1033a service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:35.772169 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e4897-10eb-4c5f-9409-4af967a1033a service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:35.817050 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:35.831054 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b2081-9451-478a-8de1-4085f9528217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:35.832676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b2081-9451-478a-8de1-4085f9528217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:35.832676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b2081-9451-478a-8de1-4085f9528217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:35.832676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b2081-9451-478a-8de1-4085f9528217 admin admin] PATH_INFO: `/v3/users/27db225222374a3ba16d7f1f15fdbf54` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:35.833772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b2081-9451-478a-8de1-4085f9528217 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27db225222374a3ba16d7f1f15fdbf54)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:35.835699 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b2081-9451-478a-8de1-4085f9528217 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:36.027208 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-054b2081-9451-478a-8de1-4085f9528217 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27db225222374a3ba16d7f1f15fdbf54'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:36.027817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-054b2081-9451-478a-8de1-4085f9528217 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27db225222374a3ba16d7f1f15fdbf54'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:36.035847 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-054b2081-9451-478a-8de1-4085f9528217 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27db225222374a3ba16d7f1f15fdbf54'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:36.036480 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-054b2081-9451-478a-8de1-4085f9528217 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27db225222374a3ba16d7f1f15fdbf54'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:36.049366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-054b2081-9451-478a-8de1-4085f9528217 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27db225222374a3ba16d7f1f15fdbf54'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:36.115179 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1397/2820] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:50:35 2022] DELETE /identity/v3/users/27db225222374a3ba16d7f1f15fdbf54 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:36.539054 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edba4e2-0814-4165-9072-d8f7e3fc4808 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:36.540587 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5edba4e2-0814-4165-9072-d8f7e3fc4808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:36.553991 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edba4e2-0814-4165-9072-d8f7e3fc4808 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:36.557791 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:36.616974 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a5f51-2f1a-497e-be74-4df1c5e5d095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:36.617879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a5f51-2f1a-497e-be74-4df1c5e5d095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:36.618158 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a5f51-2f1a-497e-be74-4df1c5e5d095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:36.618429 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a5f51-2f1a-497e-be74-4df1c5e5d095 admin admin] PATH_INFO: `/v3/projects/a9fe3ebcd02e46f181e59f8423f25e7f` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:36.619598 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a5f51-2f1a-497e-be74-4df1c5e5d095 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9fe3ebcd02e46f181e59f8423f25e7f)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:36.621556 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a5f51-2f1a-497e-be74-4df1c5e5d095 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:36.738535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2a6a5f51-2f1a-497e-be74-4df1c5e5d095 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9fe3ebcd02e46f181e59f8423f25e7f', 'request_id': 'req-2a6a5f51-2f1a-497e-be74-4df1c5e5d095'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:36.744669 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2a6a5f51-2f1a-497e-be74-4df1c5e5d095 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9fe3ebcd02e46f181e59f8423f25e7f', 'request_id': 'req-2a6a5f51-2f1a-497e-be74-4df1c5e5d095'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:36.822220 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2a6a5f51-2f1a-497e-be74-4df1c5e5d095 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9fe3ebcd02e46f181e59f8423f25e7f', 'request_id': 'req-2a6a5f51-2f1a-497e-be74-4df1c5e5d095'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:36.833821 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2a6a5f51-2f1a-497e-be74-4df1c5e5d095 admin admin] The token cache is being invalidate because project a9fe3ebcd02e46f181e59f8423f25e7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:50:36.834107 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2a6a5f51-2f1a-497e-be74-4df1c5e5d095 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:36.834753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2a6a5f51-2f1a-497e-be74-4df1c5e5d095 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:36.836295 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1424/2821] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:50:36 2022] DELETE /identity/v3/projects/a9fe3ebcd02e46f181e59f8423f25e7f => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:36.841250 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba5fad-0139-4873-ac6b-f615e485bd00 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:36.841890 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba5fad-0139-4873-ac6b-f615e485bd00 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:36.841994 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba5fad-0139-4873-ac6b-f615e485bd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:36.860853 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-38ba5fad-0139-4873-ac6b-f615e485bd00 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:36.862104 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-38ba5fad-0139-4873-ac6b-f615e485bd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:36.923094 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1398/2822] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:36.930566 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e29253-8cf1-4597-8b60-5e95e81b814d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:36.930855 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e29253-8cf1-4597-8b60-5e95e81b814d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:36.931137 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e29253-8cf1-4597-8b60-5e95e81b814d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:36.950325 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-06e29253-8cf1-4597-8b60-5e95e81b814d None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:36.951540 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-06e29253-8cf1-4597-8b60-5e95e81b814d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:36.960023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1425/2823] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:36.964402 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba5fad-0139-4873-ac6b-f615e485bd00 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:36.975247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba5fad-0139-4873-ac6b-f615e485bd00 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:36.980690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e33ec-ed70-42d2-b2e3-7bfe031bd6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:36.981430 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e33ec-ed70-42d2-b2e3-7bfe031bd6f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:36.981707 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e33ec-ed70-42d2-b2e3-7bfe031bd6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:36.981977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e33ec-ed70-42d2-b2e3-7bfe031bd6f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:36.982692 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e33ec-ed70-42d2-b2e3-7bfe031bd6f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:50:36.982995 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e33ec-ed70-42d2-b2e3-7bfe031bd6f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:36.985175 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e33ec-ed70-42d2-b2e3-7bfe031bd6f4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:36.997658 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1399/2824] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:50:37.002388 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e29253-8cf1-4597-8b60-5e95e81b814d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:37.012352 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e29253-8cf1-4597-8b60-5e95e81b814d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:37.018233 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083e53c-17e5-4c3b-adde-e31450864658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:37.018890 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083e53c-17e5-4c3b-adde-e31450864658 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:37.019174 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083e53c-17e5-4c3b-adde-e31450864658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:37.019448 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083e53c-17e5-4c3b-adde-e31450864658 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:37.020140 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083e53c-17e5-4c3b-adde-e31450864658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:37.022127 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083e53c-17e5-4c3b-adde-e31450864658 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:37.133747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1426/2825] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:50:37 2022] POST /identity/v3/projects => generated 407 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:50:37.137840 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e33ec-ed70-42d2-b2e3-7bfe031bd6f4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:37.148275 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e33ec-ed70-42d2-b2e3-7bfe031bd6f4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:37.199008 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:37.207828 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649c715-93a9-4f3a-a4ac-53bdce2267b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:37.215530 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649c715-93a9-4f3a-a4ac-53bdce2267b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:37.215530 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649c715-93a9-4f3a-a4ac-53bdce2267b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:37.215530 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649c715-93a9-4f3a-a4ac-53bdce2267b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:37.215530 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649c715-93a9-4f3a-a4ac-53bdce2267b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:37.215530 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649c715-93a9-4f3a-a4ac-53bdce2267b8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:37.292964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1400/2826] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:50:37 2022] POST /identity/v3/users => generated 304 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:50:37.296899 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083e53c-17e5-4c3b-adde-e31450864658 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:37.308033 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083e53c-17e5-4c3b-adde-e31450864658 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:37.312649 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03374187-401d-439b-a598-ce1c37c3c912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:37.313414 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03374187-401d-439b-a598-ce1c37c3c912 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:37.313701 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03374187-401d-439b-a598-ce1c37c3c912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:37.314510 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03374187-401d-439b-a598-ce1c37c3c912 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:37.315295 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03374187-401d-439b-a598-ce1c37c3c912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:37.316885 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03374187-401d-439b-a598-ce1c37c3c912 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:37.327373 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1427/2827] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:50:37.331996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649c715-93a9-4f3a-a4ac-53bdce2267b8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:37.342161 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649c715-93a9-4f3a-a4ac-53bdce2267b8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:37.346884 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c93a5-3d81-45c8-a612-644f4196d30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:37.347599 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c93a5-3d81-45c8-a612-644f4196d30c admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:37.347877 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c93a5-3d81-45c8-a612-644f4196d30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:37.348148 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c93a5-3d81-45c8-a612-644f4196d30c admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:37.348892 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c93a5-3d81-45c8-a612-644f4196d30c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:37.350526 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c93a5-3d81-45c8-a612-644f4196d30c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:37.361517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1401/2828] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:50:37.365771 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03374187-401d-439b-a598-ce1c37c3c912 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:37.375837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03374187-401d-439b-a598-ce1c37c3c912 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:37.381164 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0165ba-14d5-43bf-8d18-554ad4cd4190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:37.382075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0165ba-14d5-43bf-8d18-554ad4cd4190 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:37.382355 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0165ba-14d5-43bf-8d18-554ad4cd4190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:37.382632 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0165ba-14d5-43bf-8d18-554ad4cd4190 admin admin] PATH_INFO: `/v3/projects/eee4a3d987234799b3b9510c126e2225/users/38dfbc39228b4fe5a65534146d2288d3/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:37.401842 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0165ba-14d5-43bf-8d18-554ad4cd4190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eee4a3d987234799b3b9510c126e2225, user_id=38dfbc39228b4fe5a65534146d2288d3, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:37.404428 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0165ba-14d5-43bf-8d18-554ad4cd4190 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:37.498619 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1428/2829] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:50:37 2022] PUT /identity/v3/projects/eee4a3d987234799b3b9510c126e2225/users/38dfbc39228b4fe5a65534146d2288d3/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:37.509910 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c9df9-5ed0-4017-b7a0-afa50672e959 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:37.510347 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c9df9-5ed0-4017-b7a0-afa50672e959 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:37.510629 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c9df9-5ed0-4017-b7a0-afa50672e959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:37.538508 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-849c9df9-5ed0-4017-b7a0-afa50672e959 None None] MFA Rules not processed for user `38dfbc39228b4fe5a65534146d2288d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:37.539729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-849c9df9-5ed0-4017-b7a0-afa50672e959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:37.607086 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1402/2830] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:37 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:37.612109 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0165ba-14d5-43bf-8d18-554ad4cd4190 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:37.646780 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:37.674760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0165ba-14d5-43bf-8d18-554ad4cd4190 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:37.679646 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca0004-4a87-400e-8733-d8b34d0d914b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:37.680548 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca0004-4a87-400e-8733-d8b34d0d914b admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:37.680898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca0004-4a87-400e-8733-d8b34d0d914b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:37.681596 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca0004-4a87-400e-8733-d8b34d0d914b admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:37.682334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca0004-4a87-400e-8733-d8b34d0d914b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:37.684495 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca0004-4a87-400e-8733-d8b34d0d914b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:37.785980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1429/2831] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:50:37 2022] POST /identity/v3/projects => generated 409 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:50:37.790184 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c9df9-5ed0-4017-b7a0-afa50672e959 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:37.802009 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c9df9-5ed0-4017-b7a0-afa50672e959 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:37.860077 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b33132-1c0b-4c2d-b644-53d5bd70492d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:37.861761 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b33132-1c0b-4c2d-b644-53d5bd70492d admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:37.861761 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b33132-1c0b-4c2d-b644-53d5bd70492d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:37.862073 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b33132-1c0b-4c2d-b644-53d5bd70492d admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:37.862567 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b33132-1c0b-4c2d-b644-53d5bd70492d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:37.864501 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b33132-1c0b-4c2d-b644-53d5bd70492d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:37.938411 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1403/2832] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:50:37 2022] POST /identity/v3/users => generated 311 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:50:37.942990 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca0004-4a87-400e-8733-d8b34d0d914b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:37.953637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca0004-4a87-400e-8733-d8b34d0d914b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:37.958847 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee2cef-94ec-40f1-801e-691768b79da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:37.959515 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee2cef-94ec-40f1-801e-691768b79da6 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:37.959804 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee2cef-94ec-40f1-801e-691768b79da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:37.960081 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee2cef-94ec-40f1-801e-691768b79da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:37.960811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee2cef-94ec-40f1-801e-691768b79da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:37.962463 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee2cef-94ec-40f1-801e-691768b79da6 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:37.973974 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1430/2833] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:50:37.978456 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b33132-1c0b-4c2d-b644-53d5bd70492d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:37.989431 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b33132-1c0b-4c2d-b644-53d5bd70492d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:37.994041 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e1b20-3787-4cd7-ab8a-a5d51b3b9cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:37.994928 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e1b20-3787-4cd7-ab8a-a5d51b3b9cf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:37.995208 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e1b20-3787-4cd7-ab8a-a5d51b3b9cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:37.995482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e1b20-3787-4cd7-ab8a-a5d51b3b9cf6 admin admin] PATH_INFO: `/v3/projects/c84195dff76b473c94718b7040ca374b/users/88494339c7ad4a12ad7adf1dc0db086b/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:38.010729 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e1b20-3787-4cd7-ab8a-a5d51b3b9cf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84195dff76b473c94718b7040ca374b, user_id=88494339c7ad4a12ad7adf1dc0db086b, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:38.013464 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e1b20-3787-4cd7-ab8a-a5d51b3b9cf6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:38.092667 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1404/2834] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:50:37 2022] PUT /identity/v3/projects/c84195dff76b473c94718b7040ca374b/users/88494339c7ad4a12ad7adf1dc0db086b/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 08:50:38.097550 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee2cef-94ec-40f1-801e-691768b79da6 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:38.108227 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee2cef-94ec-40f1-801e-691768b79da6 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:38.166324 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2741a7-7bca-4d6a-abde-55ae7178ad6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:38.166991 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2741a7-7bca-4d6a-abde-55ae7178ad6c admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:38.167270 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2741a7-7bca-4d6a-abde-55ae7178ad6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:38.167542 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2741a7-7bca-4d6a-abde-55ae7178ad6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:38.168265 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2741a7-7bca-4d6a-abde-55ae7178ad6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:38.169917 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2741a7-7bca-4d6a-abde-55ae7178ad6c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:38.181803 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1431/2835] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:50:38.186599 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e1b20-3787-4cd7-ab8a-a5d51b3b9cf6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:38.200339 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e1b20-3787-4cd7-ab8a-a5d51b3b9cf6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:38.207959 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f43615-6cc0-4260-acfe-7c3c820c344c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:38.208853 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f43615-6cc0-4260-acfe-7c3c820c344c admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:38.209163 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f43615-6cc0-4260-acfe-7c3c820c344c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:38.209436 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f43615-6cc0-4260-acfe-7c3c820c344c admin admin] PATH_INFO: `/v3/projects/c84195dff76b473c94718b7040ca374b/users/88494339c7ad4a12ad7adf1dc0db086b/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:38.211417 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f43615-6cc0-4260-acfe-7c3c820c344c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84195dff76b473c94718b7040ca374b, user_id=88494339c7ad4a12ad7adf1dc0db086b, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:38.214183 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f43615-6cc0-4260-acfe-7c3c820c344c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:38.225364 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1405/2836] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:50:38 2022] PUT /identity/v3/projects/c84195dff76b473c94718b7040ca374b/users/88494339c7ad4a12ad7adf1dc0db086b/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:38.237637 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bce43-ae25-49ab-af3b-f5e061de8ebb None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:38.237942 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bce43-ae25-49ab-af3b-f5e061de8ebb None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:38.238221 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bce43-ae25-49ab-af3b-f5e061de8ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:38.265793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c50bce43-ae25-49ab-af3b-f5e061de8ebb None None] MFA Rules not processed for user `88494339c7ad4a12ad7adf1dc0db086b`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:38.267004 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c50bce43-ae25-49ab-af3b-f5e061de8ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:38.340313 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1432/2837] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:38 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:38.351022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f43615-6cc0-4260-acfe-7c3c820c344c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:38.352676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d8f43615-6cc0-4260-acfe-7c3c820c344c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:38.367331 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:38.412392 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f43615-6cc0-4260-acfe-7c3c820c344c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:38.416120 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f0a35-14d7-46cb-a23e-dab0ae0179f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:38.416767 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f0a35-14d7-46cb-a23e-dab0ae0179f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:38.417084 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f0a35-14d7-46cb-a23e-dab0ae0179f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:38.417366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f0a35-14d7-46cb-a23e-dab0ae0179f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:38.427632 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f0a35-14d7-46cb-a23e-dab0ae0179f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:38.429109 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f0a35-14d7-46cb-a23e-dab0ae0179f9 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:38.440055 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1406/2838] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:50:38 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:50:38.748236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50bce43-ae25-49ab-af3b-f5e061de8ebb None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:38.759118 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50bce43-ae25-49ab-af3b-f5e061de8ebb None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:38.762810 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81980c5e-a374-413e-9862-1f4c89adc1f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:38.763455 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81980c5e-a374-413e-9862-1f4c89adc1f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:38.763738 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81980c5e-a374-413e-9862-1f4c89adc1f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:38.764091 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81980c5e-a374-413e-9862-1f4c89adc1f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:38.791620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:38.825430 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81980c5e-a374-413e-9862-1f4c89adc1f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:38.826927 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81980c5e-a374-413e-9862-1f4c89adc1f5 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:38.836941 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1433/2839] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:50:38 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:50:48.435151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=20 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:48.707357 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f0a35-14d7-46cb-a23e-dab0ae0179f9 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:48.770635 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f0a35-14d7-46cb-a23e-dab0ae0179f9 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:48.775749 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92827ff9-0895-496a-9afe-85a1389ec58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:48.776775 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92827ff9-0895-496a-9afe-85a1389ec58b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:48.777065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92827ff9-0895-496a-9afe-85a1389ec58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:48.777354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92827ff9-0895-496a-9afe-85a1389ec58b admin admin] PATH_INFO: `/v3/users/38dfbc39228b4fe5a65534146d2288d3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:48.778550 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92827ff9-0895-496a-9afe-85a1389ec58b admin admin] RBAC: Authorizing `identity:delete_user(user_id=38dfbc39228b4fe5a65534146d2288d3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:48.780489 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92827ff9-0895-496a-9afe-85a1389ec58b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:48.857155 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=15 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:48.940420 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-92827ff9-0895-496a-9afe-85a1389ec58b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38dfbc39228b4fe5a65534146d2288d3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:48.941127 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-92827ff9-0895-496a-9afe-85a1389ec58b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38dfbc39228b4fe5a65534146d2288d3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:48.948302 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-92827ff9-0895-496a-9afe-85a1389ec58b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38dfbc39228b4fe5a65534146d2288d3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:48.949428 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-92827ff9-0895-496a-9afe-85a1389ec58b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38dfbc39228b4fe5a65534146d2288d3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:48.961774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-92827ff9-0895-496a-9afe-85a1389ec58b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38dfbc39228b4fe5a65534146d2288d3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:49.029178 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1407/2840] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:50:48 2022] DELETE /identity/v3/users/38dfbc39228b4fe5a65534146d2288d3 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:49.033901 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab59c17-4ccb-43fb-8d36-06ab8bd42da4 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:49.034215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab59c17-4ccb-43fb-8d36-06ab8bd42da4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:49.035401 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab59c17-4ccb-43fb-8d36-06ab8bd42da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:49.052173 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-2ab59c17-4ccb-43fb-8d36-06ab8bd42da4 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:49.053438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2ab59c17-4ccb-43fb-8d36-06ab8bd42da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:49.112482 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1434/2841] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:49.122705 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92827ff9-0895-496a-9afe-85a1389ec58b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:49.124332 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-92827ff9-0895-496a-9afe-85a1389ec58b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:49.182482 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92827ff9-0895-496a-9afe-85a1389ec58b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:49.186202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4fcec-5cd3-444f-98ea-cf66e66a62b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:49.186900 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4fcec-5cd3-444f-98ea-cf66e66a62b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:49.187184 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4fcec-5cd3-444f-98ea-cf66e66a62b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:49.187462 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4fcec-5cd3-444f-98ea-cf66e66a62b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:49.197720 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4fcec-5cd3-444f-98ea-cf66e66a62b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:49.199151 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4fcec-5cd3-444f-98ea-cf66e66a62b6 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:49.209493 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1408/2842] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:50:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:50:49.653106 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab59c17-4ccb-43fb-8d36-06ab8bd42da4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:49.654545 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2ab59c17-4ccb-43fb-8d36-06ab8bd42da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:49.668020 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab59c17-4ccb-43fb-8d36-06ab8bd42da4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:49.672707 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ff57c-3788-4e90-b97e-73a2086c4226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:49.673629 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ff57c-3788-4e90-b97e-73a2086c4226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:49.673909 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ff57c-3788-4e90-b97e-73a2086c4226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:49.674183 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ff57c-3788-4e90-b97e-73a2086c4226 admin admin] PATH_INFO: `/v3/projects/eee4a3d987234799b3b9510c126e2225` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:49.675386 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ff57c-3788-4e90-b97e-73a2086c4226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eee4a3d987234799b3b9510c126e2225)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:49.677413 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ff57c-3788-4e90-b97e-73a2086c4226 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:49.874882 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ca1ff57c-3788-4e90-b97e-73a2086c4226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eee4a3d987234799b3b9510c126e2225', 'request_id': 'req-ca1ff57c-3788-4e90-b97e-73a2086c4226'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:49.881452 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ca1ff57c-3788-4e90-b97e-73a2086c4226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eee4a3d987234799b3b9510c126e2225', 'request_id': 'req-ca1ff57c-3788-4e90-b97e-73a2086c4226'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:49.960162 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ca1ff57c-3788-4e90-b97e-73a2086c4226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eee4a3d987234799b3b9510c126e2225', 'request_id': 'req-ca1ff57c-3788-4e90-b97e-73a2086c4226'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:49.971072 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ca1ff57c-3788-4e90-b97e-73a2086c4226 admin admin] The token cache is being invalidate because project eee4a3d987234799b3b9510c126e2225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:50:49.971360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ca1ff57c-3788-4e90-b97e-73a2086c4226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:49.971980 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ca1ff57c-3788-4e90-b97e-73a2086c4226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:49.973603 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1435/2843] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:50:49 2022] DELETE /identity/v3/projects/eee4a3d987234799b3b9510c126e2225 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:49.978026 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4fcec-5cd3-444f-98ea-cf66e66a62b6 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:49.987842 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4fcec-5cd3-444f-98ea-cf66e66a62b6 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:50.006487 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:50.044757 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb199ec-64e9-4f29-b5de-d7c5b23fca70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:50.045662 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb199ec-64e9-4f29-b5de-d7c5b23fca70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:50.045946 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb199ec-64e9-4f29-b5de-d7c5b23fca70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:50.046221 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb199ec-64e9-4f29-b5de-d7c5b23fca70 admin admin] PATH_INFO: `/v3/users/88494339c7ad4a12ad7adf1dc0db086b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:50.047400 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb199ec-64e9-4f29-b5de-d7c5b23fca70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88494339c7ad4a12ad7adf1dc0db086b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:50.049922 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb199ec-64e9-4f29-b5de-d7c5b23fca70 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:50.191166 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8fb199ec-64e9-4f29-b5de-d7c5b23fca70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88494339c7ad4a12ad7adf1dc0db086b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:50.191812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8fb199ec-64e9-4f29-b5de-d7c5b23fca70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88494339c7ad4a12ad7adf1dc0db086b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:50.200812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8fb199ec-64e9-4f29-b5de-d7c5b23fca70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88494339c7ad4a12ad7adf1dc0db086b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:50.201497 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8fb199ec-64e9-4f29-b5de-d7c5b23fca70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88494339c7ad4a12ad7adf1dc0db086b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:50.213961 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8fb199ec-64e9-4f29-b5de-d7c5b23fca70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88494339c7ad4a12ad7adf1dc0db086b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:50.297466 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1409/2844] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:50:49 2022] DELETE /identity/v3/users/88494339c7ad4a12ad7adf1dc0db086b => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:50.739027 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ff57c-3788-4e90-b97e-73a2086c4226 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:50.740501 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ca1ff57c-3788-4e90-b97e-73a2086c4226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:50.754677 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ff57c-3788-4e90-b97e-73a2086c4226 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:50.819775 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5e7da-625c-476d-8b4b-232af5386802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:50.820649 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5e7da-625c-476d-8b4b-232af5386802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:50.820928 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5e7da-625c-476d-8b4b-232af5386802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:50.821230 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5e7da-625c-476d-8b4b-232af5386802 admin admin] PATH_INFO: `/v3/projects/c84195dff76b473c94718b7040ca374b` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:50.822443 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5e7da-625c-476d-8b4b-232af5386802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c84195dff76b473c94718b7040ca374b)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:50.824338 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5e7da-625c-476d-8b4b-232af5386802 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:50.922276 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:50.938879 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6fd5e7da-625c-476d-8b4b-232af5386802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c84195dff76b473c94718b7040ca374b', 'request_id': 'req-6fd5e7da-625c-476d-8b4b-232af5386802'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:50.945632 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6fd5e7da-625c-476d-8b4b-232af5386802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c84195dff76b473c94718b7040ca374b', 'request_id': 'req-6fd5e7da-625c-476d-8b4b-232af5386802'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:51.011749 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6fd5e7da-625c-476d-8b4b-232af5386802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c84195dff76b473c94718b7040ca374b', 'request_id': 'req-6fd5e7da-625c-476d-8b4b-232af5386802'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:51.022982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6fd5e7da-625c-476d-8b4b-232af5386802 admin admin] The token cache is being invalidate because project c84195dff76b473c94718b7040ca374b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:50:51.023270 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6fd5e7da-625c-476d-8b4b-232af5386802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:51.023898 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6fd5e7da-625c-476d-8b4b-232af5386802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:51.025786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1436/2845] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:50:50 2022] DELETE /identity/v3/projects/c84195dff76b473c94718b7040ca374b => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:51.030756 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5ddbc-b674-4c47-a296-63fe1d76c578 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:51.031276 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5ddbc-b674-4c47-a296-63fe1d76c578 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:51.031378 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5ddbc-b674-4c47-a296-63fe1d76c578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:51.059769 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-67f5ddbc-b674-4c47-a296-63fe1d76c578 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:51.061060 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-67f5ddbc-b674-4c47-a296-63fe1d76c578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:51.121530 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1410/2846] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:51.129201 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc27d0a-0f57-4c07-b4e5-56c0ba83704a None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:51.129490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc27d0a-0f57-4c07-b4e5-56c0ba83704a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:51.129770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc27d0a-0f57-4c07-b4e5-56c0ba83704a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:51.148163 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-dfc27d0a-0f57-4c07-b4e5-56c0ba83704a None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:51.149392 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-dfc27d0a-0f57-4c07-b4e5-56c0ba83704a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:51.157635 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1437/2847] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:51.162432 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5ddbc-b674-4c47-a296-63fe1d76c578 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:51.172120 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5ddbc-b674-4c47-a296-63fe1d76c578 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:51.176832 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058bdf5-b13d-4f85-b6ea-e02cdaa8b90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:51.177532 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058bdf5-b13d-4f85-b6ea-e02cdaa8b90d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:51.177814 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058bdf5-b13d-4f85-b6ea-e02cdaa8b90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:51.178089 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058bdf5-b13d-4f85-b6ea-e02cdaa8b90d admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:51.178811 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058bdf5-b13d-4f85-b6ea-e02cdaa8b90d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:50:51.179122 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058bdf5-b13d-4f85-b6ea-e02cdaa8b90d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:51.180725 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058bdf5-b13d-4f85-b6ea-e02cdaa8b90d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:51.192738 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1411/2848] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 08:50:51.197025 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc27d0a-0f57-4c07-b4e5-56c0ba83704a None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:51.206656 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc27d0a-0f57-4c07-b4e5-56c0ba83704a None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:51.211285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694059c-bf24-4ae7-8a5b-0dcf6a34dcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:51.211950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694059c-bf24-4ae7-8a5b-0dcf6a34dcb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:51.212209 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694059c-bf24-4ae7-8a5b-0dcf6a34dcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:51.212572 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694059c-bf24-4ae7-8a5b-0dcf6a34dcb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:51.213210 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694059c-bf24-4ae7-8a5b-0dcf6a34dcb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:51.215113 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694059c-bf24-4ae7-8a5b-0dcf6a34dcb6 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:51.309031 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1438/2849] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:50:51 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:50:51.313130 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058bdf5-b13d-4f85-b6ea-e02cdaa8b90d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:51.323690 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058bdf5-b13d-4f85-b6ea-e02cdaa8b90d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:51.341740 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:51.381187 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c3e5e-7fbc-4919-a076-24a8f45176bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:51.381852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c3e5e-7fbc-4919-a076-24a8f45176bb admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:51.382131 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c3e5e-7fbc-4919-a076-24a8f45176bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:51.382400 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c3e5e-7fbc-4919-a076-24a8f45176bb admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:51.383079 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c3e5e-7fbc-4919-a076-24a8f45176bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:51.384982 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c3e5e-7fbc-4919-a076-24a8f45176bb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:51.463267 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1412/2850] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:50:51 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:50:51.467532 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694059c-bf24-4ae7-8a5b-0dcf6a34dcb6 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:51.478495 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694059c-bf24-4ae7-8a5b-0dcf6a34dcb6 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:51.483496 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ff103-8aa0-48fd-aa7e-51eba3c3b61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:51.484350 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ff103-8aa0-48fd-aa7e-51eba3c3b61d admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:51.484459 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ff103-8aa0-48fd-aa7e-51eba3c3b61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:51.484747 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ff103-8aa0-48fd-aa7e-51eba3c3b61d admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:51.485507 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ff103-8aa0-48fd-aa7e-51eba3c3b61d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:51.487715 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ff103-8aa0-48fd-aa7e-51eba3c3b61d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:51.498610 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1439/2851] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:50:51.502592 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c3e5e-7fbc-4919-a076-24a8f45176bb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:51.512741 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c3e5e-7fbc-4919-a076-24a8f45176bb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:51.517361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c2219-ee3b-43b9-b2d4-0c200dccc5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:51.518037 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c2219-ee3b-43b9-b2d4-0c200dccc5c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:51.518321 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c2219-ee3b-43b9-b2d4-0c200dccc5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:51.518598 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c2219-ee3b-43b9-b2d4-0c200dccc5c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:51.519330 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c2219-ee3b-43b9-b2d4-0c200dccc5c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:51.520934 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c2219-ee3b-43b9-b2d4-0c200dccc5c9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:51.531523 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1413/2852] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:50:51.536376 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ff103-8aa0-48fd-aa7e-51eba3c3b61d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:51.546493 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ff103-8aa0-48fd-aa7e-51eba3c3b61d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:51.552172 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d184ad-80c4-47aa-aa85-df285368f49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:51.553189 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d184ad-80c4-47aa-aa85-df285368f49d admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:51.553472 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d184ad-80c4-47aa-aa85-df285368f49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:51.553748 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d184ad-80c4-47aa-aa85-df285368f49d admin admin] PATH_INFO: `/v3/projects/cfc019a7d0d74806981195b40f2d7167/users/ebbc283936634032bab2164c1cb42330/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:51.571182 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d184ad-80c4-47aa-aa85-df285368f49d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfc019a7d0d74806981195b40f2d7167, user_id=ebbc283936634032bab2164c1cb42330, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:51.573845 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d184ad-80c4-47aa-aa85-df285368f49d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:51.647391 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1440/2853] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:50:51 2022] PUT /identity/v3/projects/cfc019a7d0d74806981195b40f2d7167/users/ebbc283936634032bab2164c1cb42330/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:51.659340 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f51259-18d0-4c9f-9f07-98cf4f46570c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:51.659631 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f51259-18d0-4c9f-9f07-98cf4f46570c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:51.659912 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f51259-18d0-4c9f-9f07-98cf4f46570c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:51.686398 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e4f51259-18d0-4c9f-9f07-98cf4f46570c None None] MFA Rules not processed for user `ebbc283936634032bab2164c1cb42330`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:51.687614 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e4f51259-18d0-4c9f-9f07-98cf4f46570c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:51.755017 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1414/2854] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:51.766194 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d184ad-80c4-47aa-aa85-df285368f49d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:51.768557 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d8d184ad-80c4-47aa-aa85-df285368f49d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:51.849447 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d184ad-80c4-47aa-aa85-df285368f49d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:51.853057 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26765f5a-df13-4a30-acc7-2e2fc5c17c43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:51.853698 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26765f5a-df13-4a30-acc7-2e2fc5c17c43 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:51.853977 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26765f5a-df13-4a30-acc7-2e2fc5c17c43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:51.854249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26765f5a-df13-4a30-acc7-2e2fc5c17c43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:51.863288 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26765f5a-df13-4a30-acc7-2e2fc5c17c43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:51.864769 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26765f5a-df13-4a30-acc7-2e2fc5c17c43 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:51.873823 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1441/2855] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:50:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:50:51.893927 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f51259-18d0-4c9f-9f07-98cf4f46570c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:51.956904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f51259-18d0-4c9f-9f07-98cf4f46570c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:51.961449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a3ecc-d287-4ea8-9bec-ed143e40b361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:51.962365 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a3ecc-d287-4ea8-9bec-ed143e40b361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:51.962655 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a3ecc-d287-4ea8-9bec-ed143e40b361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:51.962924 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a3ecc-d287-4ea8-9bec-ed143e40b361 admin admin] PATH_INFO: `/v3/users/ebbc283936634032bab2164c1cb42330` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:51.964117 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a3ecc-d287-4ea8-9bec-ed143e40b361 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebbc283936634032bab2164c1cb42330)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:51.966070 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a3ecc-d287-4ea8-9bec-ed143e40b361 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:52.144034 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:52.157232 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0d1a3ecc-d287-4ea8-9bec-ed143e40b361 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebbc283936634032bab2164c1cb42330'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:52.157894 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0d1a3ecc-d287-4ea8-9bec-ed143e40b361 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebbc283936634032bab2164c1cb42330'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:52.164709 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0d1a3ecc-d287-4ea8-9bec-ed143e40b361 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebbc283936634032bab2164c1cb42330'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:52.165360 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0d1a3ecc-d287-4ea8-9bec-ed143e40b361 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebbc283936634032bab2164c1cb42330'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:52.177040 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0d1a3ecc-d287-4ea8-9bec-ed143e40b361 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebbc283936634032bab2164c1cb42330'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:52.263058 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1415/2856] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:50:51 2022] DELETE /identity/v3/users/ebbc283936634032bab2164c1cb42330 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:52.267638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428593a-c67e-4cbe-961b-71d64dd0bcde None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:52.268115 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428593a-c67e-4cbe-961b-71d64dd0bcde None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:52.268215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428593a-c67e-4cbe-961b-71d64dd0bcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:52.286467 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-8428593a-c67e-4cbe-961b-71d64dd0bcde None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:52.287675 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8428593a-c67e-4cbe-961b-71d64dd0bcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:52.317834 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:52.348095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1442/2857] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:52.357680 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a3ecc-d287-4ea8-9bec-ed143e40b361 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:52.359079 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0d1a3ecc-d287-4ea8-9bec-ed143e40b361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:52.417135 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a3ecc-d287-4ea8-9bec-ed143e40b361 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:52.420790 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc1b9a-fc5d-42bc-b392-7e9a8c184c92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:52.423285 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc1b9a-fc5d-42bc-b392-7e9a8c184c92 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:52.425496 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc1b9a-fc5d-42bc-b392-7e9a8c184c92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:52.425496 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc1b9a-fc5d-42bc-b392-7e9a8c184c92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:52.433769 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc1b9a-fc5d-42bc-b392-7e9a8c184c92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:52.435254 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc1b9a-fc5d-42bc-b392-7e9a8c184c92 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:52.445097 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1416/2858] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:50:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:50:52.869515 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428593a-c67e-4cbe-961b-71d64dd0bcde None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:52.871059 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8428593a-c67e-4cbe-961b-71d64dd0bcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:52.884966 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428593a-c67e-4cbe-961b-71d64dd0bcde None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:52.889700 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b57a1e8-e2c5-488a-bca5-d315bf776d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:52.890676 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57a1e8-e2c5-488a-bca5-d315bf776d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:52.890959 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57a1e8-e2c5-488a-bca5-d315bf776d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:52.891469 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57a1e8-e2c5-488a-bca5-d315bf776d16 admin admin] PATH_INFO: `/v3/projects/cfc019a7d0d74806981195b40f2d7167` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:52.893451 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b57a1e8-e2c5-488a-bca5-d315bf776d16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfc019a7d0d74806981195b40f2d7167)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:52.895285 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b57a1e8-e2c5-488a-bca5-d315bf776d16 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:53.024001 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0b57a1e8-e2c5-488a-bca5-d315bf776d16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfc019a7d0d74806981195b40f2d7167', 'request_id': 'req-0b57a1e8-e2c5-488a-bca5-d315bf776d16'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:53.030330 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0b57a1e8-e2c5-488a-bca5-d315bf776d16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfc019a7d0d74806981195b40f2d7167', 'request_id': 'req-0b57a1e8-e2c5-488a-bca5-d315bf776d16'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:53.097406 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0b57a1e8-e2c5-488a-bca5-d315bf776d16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfc019a7d0d74806981195b40f2d7167', 'request_id': 'req-0b57a1e8-e2c5-488a-bca5-d315bf776d16'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:53.108098 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0b57a1e8-e2c5-488a-bca5-d315bf776d16 admin admin] The token cache is being invalidate because project cfc019a7d0d74806981195b40f2d7167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:50:53.108387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0b57a1e8-e2c5-488a-bca5-d315bf776d16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:53.109049 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-0b57a1e8-e2c5-488a-bca5-d315bf776d16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:50:53.110611 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1443/2859] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:50:52 2022] DELETE /identity/v3/projects/cfc019a7d0d74806981195b40f2d7167 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:53.116165 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a2089-35a2-4ada-a4ed-0a43d808ca77 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:53.116454 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a2089-35a2-4ada-a4ed-0a43d808ca77 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:53.116732 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a2089-35a2-4ada-a4ed-0a43d808ca77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:53.134904 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0f7a2089-35a2-4ada-a4ed-0a43d808ca77 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:53.136104 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0f7a2089-35a2-4ada-a4ed-0a43d808ca77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:53.196198 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1417/2860] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 08:50:53.203782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fbec3-46e4-46be-b355-4c021b0b5aff None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:53.204082 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fbec3-46e4-46be-b355-4c021b0b5aff None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:53.204508 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fbec3-46e4-46be-b355-4c021b0b5aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:53.223802 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ec6fbec3-46e4-46be-b355-4c021b0b5aff None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:53.225024 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ec6fbec3-46e4-46be-b355-4c021b0b5aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:53.232933 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1444/2861] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:53.237154 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a2089-35a2-4ada-a4ed-0a43d808ca77 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:53.246860 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a2089-35a2-4ada-a4ed-0a43d808ca77 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:53.251632 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbff04-c9d1-4a4b-b899-86fe6781b38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:53.252316 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbff04-c9d1-4a4b-b899-86fe6781b38f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:53.252615 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbff04-c9d1-4a4b-b899-86fe6781b38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:53.252874 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbff04-c9d1-4a4b-b899-86fe6781b38f admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:53.253817 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbff04-c9d1-4a4b-b899-86fe6781b38f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:50:53.254132 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbff04-c9d1-4a4b-b899-86fe6781b38f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:53.255772 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbff04-c9d1-4a4b-b899-86fe6781b38f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:53.268219 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1418/2862] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:50:53.272694 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fbec3-46e4-46be-b355-4c021b0b5aff None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:53.282435 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fbec3-46e4-46be-b355-4c021b0b5aff None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:53.287601 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94249837-d75c-456f-a573-5e50f2229646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:53.288249 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94249837-d75c-456f-a573-5e50f2229646 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:53.288535 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94249837-d75c-456f-a573-5e50f2229646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:53.288814 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94249837-d75c-456f-a573-5e50f2229646 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:53.289553 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94249837-d75c-456f-a573-5e50f2229646 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:53.291568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94249837-d75c-456f-a573-5e50f2229646 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:53.372647 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1445/2863] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:50:53 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:50:53.376632 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbff04-c9d1-4a4b-b899-86fe6781b38f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:53.386656 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbff04-c9d1-4a4b-b899-86fe6781b38f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:53.420824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:53.444828 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1ed3b-27e0-49aa-995a-96bf5681f43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:53.445525 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1ed3b-27e0-49aa-995a-96bf5681f43f admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:53.445812 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1ed3b-27e0-49aa-995a-96bf5681f43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:53.446092 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1ed3b-27e0-49aa-995a-96bf5681f43f admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:53.446779 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1ed3b-27e0-49aa-995a-96bf5681f43f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:53.448706 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1ed3b-27e0-49aa-995a-96bf5681f43f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:53.555626 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1419/2864] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:50:53 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:50:53.559401 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94249837-d75c-456f-a573-5e50f2229646 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:53.570129 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94249837-d75c-456f-a573-5e50f2229646 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:53.574988 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c26b7-add3-4bec-9bbe-0602b763ab7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:53.575662 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c26b7-add3-4bec-9bbe-0602b763ab7c admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:53.575948 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c26b7-add3-4bec-9bbe-0602b763ab7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:53.576231 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c26b7-add3-4bec-9bbe-0602b763ab7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:53.576981 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c26b7-add3-4bec-9bbe-0602b763ab7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:53.578617 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c26b7-add3-4bec-9bbe-0602b763ab7c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:53.589913 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1446/2865] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:50:53.594102 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1ed3b-27e0-49aa-995a-96bf5681f43f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:53.605165 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1ed3b-27e0-49aa-995a-96bf5681f43f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:53.609824 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df12d2-4ec4-497c-8431-5fee6c2e5acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:53.610531 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df12d2-4ec4-497c-8431-5fee6c2e5acb admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:53.610813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df12d2-4ec4-497c-8431-5fee6c2e5acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:53.611090 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df12d2-4ec4-497c-8431-5fee6c2e5acb admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:53.612408 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df12d2-4ec4-497c-8431-5fee6c2e5acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:53.614041 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df12d2-4ec4-497c-8431-5fee6c2e5acb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:53.624061 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1420/2866] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:50:53.628281 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c26b7-add3-4bec-9bbe-0602b763ab7c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:53.638461 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c26b7-add3-4bec-9bbe-0602b763ab7c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:53.643139 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24728541-1391-443c-b998-a6e943fa7b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:53.644017 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24728541-1391-443c-b998-a6e943fa7b10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:53.644297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24728541-1391-443c-b998-a6e943fa7b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:53.644570 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24728541-1391-443c-b998-a6e943fa7b10 admin admin] PATH_INFO: `/v3/projects/e55eab259fe54789a8eed6824f2dd416/users/8563dc013a9e4af08aadbb3cb9a15f48/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:53.660837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24728541-1391-443c-b998-a6e943fa7b10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e55eab259fe54789a8eed6824f2dd416, user_id=8563dc013a9e4af08aadbb3cb9a15f48, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:53.663567 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24728541-1391-443c-b998-a6e943fa7b10 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:53.714344 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1447/2867] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:50:53 2022] PUT /identity/v3/projects/e55eab259fe54789a8eed6824f2dd416/users/8563dc013a9e4af08aadbb3cb9a15f48/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:50:53.725206 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1649d0-ccbe-4a31-b778-0b4edf90dbc9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:53.725498 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1649d0-ccbe-4a31-b778-0b4edf90dbc9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:53.725813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1649d0-ccbe-4a31-b778-0b4edf90dbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:53.753301 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ae1649d0-ccbe-4a31-b778-0b4edf90dbc9 None None] MFA Rules not processed for user `8563dc013a9e4af08aadbb3cb9a15f48`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:50:53.754551 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ae1649d0-ccbe-4a31-b778-0b4edf90dbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:53.817486 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1421/2868] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:50:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:50:53.828319 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24728541-1391-443c-b998-a6e943fa7b10 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:50:53.830625 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-24728541-1391-443c-b998-a6e943fa7b10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:50:53.885909 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24728541-1391-443c-b998-a6e943fa7b10 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:50:53.889564 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c5a4ba-06ce-4813-9369-c520183fc049 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85ea2962e7094f8c878d95680764c4e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:50:53.890207 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5a4ba-06ce-4813-9369-c520183fc049 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:50:53.890487 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5a4ba-06ce-4813-9369-c520183fc049 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:50:53.890765 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5a4ba-06ce-4813-9369-c520183fc049 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:50:53.897589 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:50:53.900278 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5a4ba-06ce-4813-9369-c520183fc049 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:50:53.901761 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5a4ba-06ce-4813-9369-c520183fc049 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:50:53.910719 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1448/2869] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:50:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:51:03.814189 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=49,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:51:03.981363 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:51:04.996254 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1649d0-ccbe-4a31-b778-0b4edf90dbc9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:05.061235 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1649d0-ccbe-4a31-b778-0b4edf90dbc9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:05.065782 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881958a3-1fe5-483a-9b93-04f79965650e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:05.066651 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881958a3-1fe5-483a-9b93-04f79965650e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:05.066960 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881958a3-1fe5-483a-9b93-04f79965650e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:05.067235 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881958a3-1fe5-483a-9b93-04f79965650e admin admin] PATH_INFO: `/v3/users/8563dc013a9e4af08aadbb3cb9a15f48` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:05.068434 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881958a3-1fe5-483a-9b93-04f79965650e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8563dc013a9e4af08aadbb3cb9a15f48)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:05.070555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881958a3-1fe5-483a-9b93-04f79965650e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:05.243290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-881958a3-1fe5-483a-9b93-04f79965650e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8563dc013a9e4af08aadbb3cb9a15f48'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:51:05.243881 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-881958a3-1fe5-483a-9b93-04f79965650e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8563dc013a9e4af08aadbb3cb9a15f48'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:51:05.250708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-881958a3-1fe5-483a-9b93-04f79965650e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8563dc013a9e4af08aadbb3cb9a15f48'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:51:05.251364 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-881958a3-1fe5-483a-9b93-04f79965650e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8563dc013a9e4af08aadbb3cb9a15f48'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:51:05.263886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-881958a3-1fe5-483a-9b93-04f79965650e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8563dc013a9e4af08aadbb3cb9a15f48'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:51:05.314014 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1422/2870] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:51:04 2022] DELETE /identity/v3/users/8563dc013a9e4af08aadbb3cb9a15f48 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:51:05.318679 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3d052-4176-464a-a07d-331d97f7ef70 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:05.319144 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3d052-4176-464a-a07d-331d97f7ef70 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:05.321351 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3d052-4176-464a-a07d-331d97f7ef70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:05.337567 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-f5a3d052-4176-464a-a07d-331d97f7ef70 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:51:05.338779 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f5a3d052-4176-464a-a07d-331d97f7ef70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:51:05.407827 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1449/2871] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:51:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:51:05.418506 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881958a3-1fe5-483a-9b93-04f79965650e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:05.419921 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-881958a3-1fe5-483a-9b93-04f79965650e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:51:05.477502 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881958a3-1fe5-483a-9b93-04f79965650e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:05.481170 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e14de0-301e-4857-afb1-b0f4f10640bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:05.481871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e14de0-301e-4857-afb1-b0f4f10640bb service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:05.482146 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e14de0-301e-4857-afb1-b0f4f10640bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:05.482418 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e14de0-301e-4857-afb1-b0f4f10640bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:05.492547 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e14de0-301e-4857-afb1-b0f4f10640bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:05.494076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e14de0-301e-4857-afb1-b0f4f10640bb service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:05.503996 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1423/2872] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:51:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:51:05.908841 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3d052-4176-464a-a07d-331d97f7ef70 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:05.910412 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f5a3d052-4176-464a-a07d-331d97f7ef70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:51:05.923238 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3d052-4176-464a-a07d-331d97f7ef70 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:05.927946 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0cb7c1-06b0-455a-9377-5f158b6a9ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:05.928815 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cb7c1-06b0-455a-9377-5f158b6a9ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:05.929142 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cb7c1-06b0-455a-9377-5f158b6a9ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:05.929424 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cb7c1-06b0-455a-9377-5f158b6a9ead admin admin] PATH_INFO: `/v3/projects/e55eab259fe54789a8eed6824f2dd416` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:05.930620 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0cb7c1-06b0-455a-9377-5f158b6a9ead admin admin] RBAC: Authorizing `identity:delete_project(project_id=e55eab259fe54789a8eed6824f2dd416)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:05.932657 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0cb7c1-06b0-455a-9377-5f158b6a9ead admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:06.033092 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9a0cb7c1-06b0-455a-9377-5f158b6a9ead admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e55eab259fe54789a8eed6824f2dd416', 'request_id': 'req-9a0cb7c1-06b0-455a-9377-5f158b6a9ead'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:51:06.039259 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9a0cb7c1-06b0-455a-9377-5f158b6a9ead admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e55eab259fe54789a8eed6824f2dd416', 'request_id': 'req-9a0cb7c1-06b0-455a-9377-5f158b6a9ead'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:51:06.108145 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9a0cb7c1-06b0-455a-9377-5f158b6a9ead admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e55eab259fe54789a8eed6824f2dd416', 'request_id': 'req-9a0cb7c1-06b0-455a-9377-5f158b6a9ead'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:51:06.119322 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9a0cb7c1-06b0-455a-9377-5f158b6a9ead admin admin] The token cache is being invalidate because project e55eab259fe54789a8eed6824f2dd416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:51:06.123194 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9a0cb7c1-06b0-455a-9377-5f158b6a9ead admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:51:06.123194 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9a0cb7c1-06b0-455a-9377-5f158b6a9ead admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:51:06.123194 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1450/2873] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:51:05 2022] DELETE /identity/v3/projects/e55eab259fe54789a8eed6824f2dd416 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:51:06.126838 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44e67a-7886-4fc4-ae9a-b7bfc519bda0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:06.127377 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44e67a-7886-4fc4-ae9a-b7bfc519bda0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:06.127377 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44e67a-7886-4fc4-ae9a-b7bfc519bda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:06.146488 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9e44e67a-7886-4fc4-ae9a-b7bfc519bda0 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:51:06.147701 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9e44e67a-7886-4fc4-ae9a-b7bfc519bda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:51:06.165288 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:51:06.207520 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1424/2874] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:51:06.216297 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc9d21-7cdf-407a-855d-35434abd8487 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:06.216583 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc9d21-7cdf-407a-855d-35434abd8487 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:06.216862 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc9d21-7cdf-407a-855d-35434abd8487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:06.235286 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-20cc9d21-7cdf-407a-855d-35434abd8487 None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:51:06.236470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-20cc9d21-7cdf-407a-855d-35434abd8487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:51:06.244944 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1451/2875] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:51:06.249273 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44e67a-7886-4fc4-ae9a-b7bfc519bda0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:06.259007 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44e67a-7886-4fc4-ae9a-b7bfc519bda0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:06.264116 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab3650-35ac-4ed4-acd3-afd254565064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:06.264770 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab3650-35ac-4ed4-acd3-afd254565064 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:06.265084 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab3650-35ac-4ed4-acd3-afd254565064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:06.265361 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab3650-35ac-4ed4-acd3-afd254565064 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:06.266077 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab3650-35ac-4ed4-acd3-afd254565064 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 08:51:06.266385 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab3650-35ac-4ed4-acd3-afd254565064 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:06.267986 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab3650-35ac-4ed4-acd3-afd254565064 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:06.280076 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1425/2876] 10.222.0.17 () {58 vars in 1152 bytes} [Tue Jun 7 08:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 08:51:06.284561 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc9d21-7cdf-407a-855d-35434abd8487 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:06.294378 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc9d21-7cdf-407a-855d-35434abd8487 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:06.299085 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525bfff-0bca-43a9-a671-1c805b97258d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:06.299734 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525bfff-0bca-43a9-a671-1c805b97258d admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:06.300014 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525bfff-0bca-43a9-a671-1c805b97258d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:06.300286 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525bfff-0bca-43a9-a671-1c805b97258d admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:06.300982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525bfff-0bca-43a9-a671-1c805b97258d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:06.303668 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525bfff-0bca-43a9-a671-1c805b97258d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:06.378734 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1452/2877] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:51:06 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:51:06.382930 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab3650-35ac-4ed4-acd3-afd254565064 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:06.393080 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab3650-35ac-4ed4-acd3-afd254565064 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:06.449446 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b2893-0b93-4b52-8a21-6724c893809c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:06.450117 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b2893-0b93-4b52-8a21-6724c893809c admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:06.450402 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b2893-0b93-4b52-8a21-6724c893809c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:06.450679 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b2893-0b93-4b52-8a21-6724c893809c admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:06.451397 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b2893-0b93-4b52-8a21-6724c893809c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:06.453354 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b2893-0b93-4b52-8a21-6724c893809c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:06.537947 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1426/2878] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:51:06 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:51:06.542238 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525bfff-0bca-43a9-a671-1c805b97258d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:06.553167 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525bfff-0bca-43a9-a671-1c805b97258d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:06.558145 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805749d5-b187-4a57-b21c-0fc3a4d835eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:06.558811 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805749d5-b187-4a57-b21c-0fc3a4d835eb admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:06.559095 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805749d5-b187-4a57-b21c-0fc3a4d835eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:06.559370 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805749d5-b187-4a57-b21c-0fc3a4d835eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:06.560100 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805749d5-b187-4a57-b21c-0fc3a4d835eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:06.561805 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805749d5-b187-4a57-b21c-0fc3a4d835eb admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:06.572818 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1453/2879] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 08:51:06.576919 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b2893-0b93-4b52-8a21-6724c893809c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:06.587671 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b2893-0b93-4b52-8a21-6724c893809c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:06.592417 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed9470-386d-4fc1-b9f1-021a3fb9bc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:06.593168 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed9470-386d-4fc1-b9f1-021a3fb9bc10 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:06.593452 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed9470-386d-4fc1-b9f1-021a3fb9bc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:06.593755 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed9470-386d-4fc1-b9f1-021a3fb9bc10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:06.594476 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed9470-386d-4fc1-b9f1-021a3fb9bc10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:06.596134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed9470-386d-4fc1-b9f1-021a3fb9bc10 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:06.607030 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1427/2880] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:51:06.611218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805749d5-b187-4a57-b21c-0fc3a4d835eb admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:06.621382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805749d5-b187-4a57-b21c-0fc3a4d835eb admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:06.626032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed87f1-1248-48ef-ba38-f5dcefa9f4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:06.626938 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed87f1-1248-48ef-ba38-f5dcefa9f4d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:06.627218 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed87f1-1248-48ef-ba38-f5dcefa9f4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:06.627495 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed87f1-1248-48ef-ba38-f5dcefa9f4d4 admin admin] PATH_INFO: `/v3/projects/af50419d91204a8eb8c1dca13803aa2f/users/55abc5e9e3a24da9abae9a2c44466b8f/roles/67111011e94c4e08928dd5e6b7a46af1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:06.642032 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed87f1-1248-48ef-ba38-f5dcefa9f4d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af50419d91204a8eb8c1dca13803aa2f, user_id=55abc5e9e3a24da9abae9a2c44466b8f, role_id=67111011e94c4e08928dd5e6b7a46af1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:06.644675 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed87f1-1248-48ef-ba38-f5dcefa9f4d4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:06.720279 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1454/2881] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:51:06 2022] PUT /identity/v3/projects/af50419d91204a8eb8c1dca13803aa2f/users/55abc5e9e3a24da9abae9a2c44466b8f/roles/67111011e94c4e08928dd5e6b7a46af1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:51:06.725969 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7eb3d-71c9-477d-81ba-11f19c48083f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:06.726419 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7eb3d-71c9-477d-81ba-11f19c48083f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:06.726629 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7eb3d-71c9-477d-81ba-11f19c48083f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:06.744964 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ceb7eb3d-71c9-477d-81ba-11f19c48083f None None] MFA Rules not processed for user `4ce5a782a83f41fbb386c88e6d77ba55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:51:06.746186 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ceb7eb3d-71c9-477d-81ba-11f19c48083f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:51:06.803954 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1428/2882] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:51:06.813438 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed87f1-1248-48ef-ba38-f5dcefa9f4d4 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:06.815119 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b0ed87f1-1248-48ef-ba38-f5dcefa9f4d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:51:06.836827 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:51:06.874080 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed87f1-1248-48ef-ba38-f5dcefa9f4d4 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:06.877786 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31dffa-a75c-4ad8-9da6-95f22d3cb625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:06.878512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31dffa-a75c-4ad8-9da6-95f22d3cb625 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:06.879009 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31dffa-a75c-4ad8-9da6-95f22d3cb625 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:06.879497 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31dffa-a75c-4ad8-9da6-95f22d3cb625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:06.888932 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da31dffa-a75c-4ad8-9da6-95f22d3cb625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:06.890387 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da31dffa-a75c-4ad8-9da6-95f22d3cb625 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:06.900382 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1455/2883] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:51:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:51:12.066247 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3a73f-1621-4f2b-8001-57f6278cf35c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:12.066928 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3a73f-1621-4f2b-8001-57f6278cf35c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:12.067056 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3a73f-1621-4f2b-8001-57f6278cf35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:12.095786 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-39d3a73f-1621-4f2b-8001-57f6278cf35c None None] MFA Rules not processed for user `55abc5e9e3a24da9abae9a2c44466b8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:51:12.096939 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-39d3a73f-1621-4f2b-8001-57f6278cf35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:51:12.115359 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:51:12.194881 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1429/2884] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:51:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:51:12.199591 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31dffa-a75c-4ad8-9da6-95f22d3cb625 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:12.210190 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31dffa-a75c-4ad8-9da6-95f22d3cb625 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:12.215113 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58f974-cd68-404c-ae06-23244fc79db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:12.215778 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58f974-cd68-404c-ae06-23244fc79db2 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:12.216063 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58f974-cd68-404c-ae06-23244fc79db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:12.216337 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58f974-cd68-404c-ae06-23244fc79db2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:12.217133 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58f974-cd68-404c-ae06-23244fc79db2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:12.219228 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58f974-cd68-404c-ae06-23244fc79db2 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:12.282248 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1456/2885] 10.222.0.17 () {60 vars in 1152 bytes} [Tue Jun 7 08:51:12 2022] POST /identity/v3/projects => generated 379 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:51:12.286257 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3a73f-1621-4f2b-8001-57f6278cf35c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:12.297754 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3a73f-1621-4f2b-8001-57f6278cf35c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:12.358881 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47826fd-190f-4097-9e72-fa94841f6f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:12.359555 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47826fd-190f-4097-9e72-fa94841f6f8b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:12.359834 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47826fd-190f-4097-9e72-fa94841f6f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:12.360103 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47826fd-190f-4097-9e72-fa94841f6f8b admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:12.360805 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47826fd-190f-4097-9e72-fa94841f6f8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:12.362774 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47826fd-190f-4097-9e72-fa94841f6f8b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:12.429720 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1430/2886] 10.222.0.17 () {60 vars in 1143 bytes} [Tue Jun 7 08:51:12 2022] POST /identity/v3/users => generated 296 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 08:51:12.433611 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58f974-cd68-404c-ae06-23244fc79db2 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:12.445460 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58f974-cd68-404c-ae06-23244fc79db2 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:12.450235 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cface1-16bb-497e-b0d0-2d55f25b0053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:12.450911 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cface1-16bb-497e-b0d0-2d55f25b0053 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:12.451190 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cface1-16bb-497e-b0d0-2d55f25b0053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:12.451470 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cface1-16bb-497e-b0d0-2d55f25b0053 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:12.452212 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cface1-16bb-497e-b0d0-2d55f25b0053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:12.453874 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cface1-16bb-497e-b0d0-2d55f25b0053 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:12.466751 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1457/2887] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:51:12.475123 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47826fd-190f-4097-9e72-fa94841f6f8b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:12.486139 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47826fd-190f-4097-9e72-fa94841f6f8b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:12.492829 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbeb62-1403-4150-b953-cdf134f8899d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:12.494116 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbeb62-1403-4150-b953-cdf134f8899d admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:12.494486 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbeb62-1403-4150-b953-cdf134f8899d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:12.494764 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbeb62-1403-4150-b953-cdf134f8899d admin admin] PATH_INFO: `/v3/projects/f583ed2bde8b41cf9215aee042b822dd/users/9e78d2d52cb94e519a14aba6e8705b62/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:12.512233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbbeb62-1403-4150-b953-cdf134f8899d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f583ed2bde8b41cf9215aee042b822dd, user_id=9e78d2d52cb94e519a14aba6e8705b62, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:12.516291 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbbeb62-1403-4150-b953-cdf134f8899d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:12.592918 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1431/2888] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:51:12 2022] PUT /identity/v3/projects/f583ed2bde8b41cf9215aee042b822dd/users/9e78d2d52cb94e519a14aba6e8705b62/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:51:12.596699 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cface1-16bb-497e-b0d0-2d55f25b0053 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:12.608024 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cface1-16bb-497e-b0d0-2d55f25b0053 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:12.671068 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c5c33-3120-4dc8-af86-7909eed7c1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:12.671760 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c5c33-3120-4dc8-af86-7909eed7c1a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:12.672047 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c5c33-3120-4dc8-af86-7909eed7c1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:12.672347 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c5c33-3120-4dc8-af86-7909eed7c1a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:12.673134 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c5c33-3120-4dc8-af86-7909eed7c1a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:12.674783 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c5c33-3120-4dc8-af86-7909eed7c1a8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:12.692519 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1458/2889] 10.222.0.17 () {58 vars in 1121 bytes} [Tue Jun 7 08:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 08:51:12.693914 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbeb62-1403-4150-b953-cdf134f8899d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:12.705228 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbeb62-1403-4150-b953-cdf134f8899d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:12.710613 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69314fe9-5d99-4e94-9391-a446ba94dd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:12.711651 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69314fe9-5d99-4e94-9391-a446ba94dd77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:12.711780 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69314fe9-5d99-4e94-9391-a446ba94dd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:12.712054 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69314fe9-5d99-4e94-9391-a446ba94dd77 admin admin] PATH_INFO: `/v3/projects/f583ed2bde8b41cf9215aee042b822dd/users/9e78d2d52cb94e519a14aba6e8705b62/roles/8c63c00418a346b29f7db3f87a404e70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:12.714094 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69314fe9-5d99-4e94-9391-a446ba94dd77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f583ed2bde8b41cf9215aee042b822dd, user_id=9e78d2d52cb94e519a14aba6e8705b62, role_id=8c63c00418a346b29f7db3f87a404e70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:12.716833 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69314fe9-5d99-4e94-9391-a446ba94dd77 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:12.728929 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1432/2890] 10.222.0.17 () {60 vars in 1482 bytes} [Tue Jun 7 08:51:12 2022] PUT /identity/v3/projects/f583ed2bde8b41cf9215aee042b822dd/users/9e78d2d52cb94e519a14aba6e8705b62/roles/8c63c00418a346b29f7db3f87a404e70 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:51:17.517290 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c5c33-3120-4dc8-af86-7909eed7c1a8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:17.519148 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-978c5c33-3120-4dc8-af86-7909eed7c1a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:51:17.634298 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c5c33-3120-4dc8-af86-7909eed7c1a8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:17.647522 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:51:17.720502 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd6254-8598-4b71-9cc4-dfcfd6e2190b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc0d7bd7e4547f4a8d90ffc2c2e1d03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:17.721217 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd6254-8598-4b71-9cc4-dfcfd6e2190b service placement] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:17.721568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd6254-8598-4b71-9cc4-dfcfd6e2190b service placement] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:17.721774 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd6254-8598-4b71-9cc4-dfcfd6e2190b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:17.724334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0acd6254-8598-4b71-9cc4-dfcfd6e2190b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:51:17.738409 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd6254-8598-4b71-9cc4-dfcfd6e2190b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:17.740716 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd6254-8598-4b71-9cc4-dfcfd6e2190b service placement] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:17.753456 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1459/2891] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:51:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:51:18.351007 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44ade3-a960-4730-9d87-d76e0efa948a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:18.351571 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44ade3-a960-4730-9d87-d76e0efa948a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:18.351571 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44ade3-a960-4730-9d87-d76e0efa948a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:18.383319 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-aa44ade3-a960-4730-9d87-d76e0efa948a None None] MFA Rules not processed for user `9e78d2d52cb94e519a14aba6e8705b62`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:51:18.384813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-aa44ade3-a960-4730-9d87-d76e0efa948a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:51:18.481091 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:51:18.492447 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1433/2892] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:51:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:51:18.509094 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd6254-8598-4b71-9cc4-dfcfd6e2190b service placement] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:18.510821 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0acd6254-8598-4b71-9cc4-dfcfd6e2190b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:51:18.775605 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd6254-8598-4b71-9cc4-dfcfd6e2190b service placement] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:18.781950 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae93d6c-b31b-49b6-96a1-86b5271d9531 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:18.782634 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae93d6c-b31b-49b6-96a1-86b5271d9531 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:18.782923 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae93d6c-b31b-49b6-96a1-86b5271d9531 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:18.783216 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae93d6c-b31b-49b6-96a1-86b5271d9531 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:18.848053 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae93d6c-b31b-49b6-96a1-86b5271d9531 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:18.850334 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae93d6c-b31b-49b6-96a1-86b5271d9531 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:18.860040 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1460/2893] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:51:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:51:28.673608 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:51:28.855680 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=96 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:51:40.341314 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ed18a-e064-46f6-8801-9b01ac5b9ec0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:40.341798 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ed18a-e064-46f6-8801-9b01ac5b9ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:40.341908 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ed18a-e064-46f6-8801-9b01ac5b9ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:40.359708 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f60ed18a-e064-46f6-8801-9b01ac5b9ec0 None None] MFA Rules not processed for user `55abc5e9e3a24da9abae9a2c44466b8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:51:40.360913 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f60ed18a-e064-46f6-8801-9b01ac5b9ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:51:40.368368 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1434/2894] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:51:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:51:40.678618 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae93d6c-b31b-49b6-96a1-86b5271d9531 service nova] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:40.733767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae93d6c-b31b-49b6-96a1-86b5271d9531 service nova] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:40.737277 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ca5914-5a04-4808-b39e-4e2461ea5860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:40.737910 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca5914-5a04-4808-b39e-4e2461ea5860 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:40.738186 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca5914-5a04-4808-b39e-4e2461ea5860 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:40.738457 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca5914-5a04-4808-b39e-4e2461ea5860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:40.740247 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-21ca5914-5a04-4808-b39e-4e2461ea5860 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:51:40.749611 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ca5914-5a04-4808-b39e-4e2461ea5860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:40.751058 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ca5914-5a04-4808-b39e-4e2461ea5860 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:40.759475 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1461/2895] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:51:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:51:50.354561 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:51:50.755995 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=26 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:51:57.749942 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ed18a-e064-46f6-8801-9b01ac5b9ec0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:51:57.751479 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f60ed18a-e064-46f6-8801-9b01ac5b9ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:51:57.768637 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ed18a-e064-46f6-8801-9b01ac5b9ec0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:51:57.774229 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185eed50-14d6-43a9-b7d1-74215fab2631 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:51:57.774876 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185eed50-14d6-43a9-b7d1-74215fab2631 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:51:57.775155 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185eed50-14d6-43a9-b7d1-74215fab2631 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:51:57.775431 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185eed50-14d6-43a9-b7d1-74215fab2631 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:51:57.777189 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-185eed50-14d6-43a9-b7d1-74215fab2631 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:51:57.789113 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185eed50-14d6-43a9-b7d1-74215fab2631 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:51:57.790689 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185eed50-14d6-43a9-b7d1-74215fab2631 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:51:57.801256 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1435/2896] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:51:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:52:01.304831 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ca5914-5a04-4808-b39e-4e2461ea5860 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:52:01.314157 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ca5914-5a04-4808-b39e-4e2461ea5860 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:52:01.318294 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d5701-fedd-4a5f-b843-dad5d05a40ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:52:01.318937 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d5701-fedd-4a5f-b843-dad5d05a40ad service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:52:01.319213 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d5701-fedd-4a5f-b843-dad5d05a40ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:52:01.319486 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d5701-fedd-4a5f-b843-dad5d05a40ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:52:01.329047 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d5701-fedd-4a5f-b843-dad5d05a40ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:52:01.330512 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d5701-fedd-4a5f-b843-dad5d05a40ad service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:52:01.342973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1462/2897] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:52:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:52:03.380922 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185eed50-14d6-43a9-b7d1-74215fab2631 service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:52:03.392517 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185eed50-14d6-43a9-b7d1-74215fab2631 service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:52:03.398366 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d2b98-f011-4ed1-abce-e85d998fee8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:52:03.399297 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d2b98-f011-4ed1-abce-e85d998fee8f service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:52:03.399582 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d2b98-f011-4ed1-abce-e85d998fee8f service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:52:03.399860 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d2b98-f011-4ed1-abce-e85d998fee8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:52:03.401750 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-715d2b98-f011-4ed1-abce-e85d998fee8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:52:03.415226 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d2b98-f011-4ed1-abce-e85d998fee8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:52:03.416946 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d2b98-f011-4ed1-abce-e85d998fee8f service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:52:03.429171 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1436/2898] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:52:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:52:11.335723 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=6 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:52:13.423644 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=12 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:54:00.381264 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d5701-fedd-4a5f-b843-dad5d05a40ad service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:54:00.382922 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-eb5d5701-fedd-4a5f-b843-dad5d05a40ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:54:00.394333 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d5701-fedd-4a5f-b843-dad5d05a40ad service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:54:00.398313 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172700ee-881f-4468-b695-c7a59f3f4eb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:54:00.398969 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172700ee-881f-4468-b695-c7a59f3f4eb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:54:00.399245 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172700ee-881f-4468-b695-c7a59f3f4eb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:54:00.399515 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172700ee-881f-4468-b695-c7a59f3f4eb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:54:00.401369 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-172700ee-881f-4468-b695-c7a59f3f4eb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:54:00.415180 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172700ee-881f-4468-b695-c7a59f3f4eb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:54:00.416592 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172700ee-881f-4468-b695-c7a59f3f4eb9 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:54:00.427966 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1463/2899] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:54:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:54:01.611870 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d2b98-f011-4ed1-abce-e85d998fee8f service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:54:01.621447 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d2b98-f011-4ed1-abce-e85d998fee8f service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:54:01.625686 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a6896-780c-43d2-80f8-98fa128b4039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:54:01.626329 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a6896-780c-43d2-80f8-98fa128b4039 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:54:01.626612 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a6896-780c-43d2-80f8-98fa128b4039 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:54:01.626886 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a6896-780c-43d2-80f8-98fa128b4039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:54:01.628652 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f83a6896-780c-43d2-80f8-98fa128b4039 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:54:01.644270 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83a6896-780c-43d2-80f8-98fa128b4039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:54:01.646564 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83a6896-780c-43d2-80f8-98fa128b4039 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:54:01.660349 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1437/2900] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:54:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:54:03.442422 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172700ee-881f-4468-b695-c7a59f3f4eb9 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:54:03.454494 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172700ee-881f-4468-b695-c7a59f3f4eb9 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:54:03.461228 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790b06b-27a6-4323-ba29-eb69ce1dabd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:54:03.461957 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790b06b-27a6-4323-ba29-eb69ce1dabd9 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:54:03.462313 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790b06b-27a6-4323-ba29-eb69ce1dabd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:54:03.462596 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790b06b-27a6-4323-ba29-eb69ce1dabd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:54:03.464417 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2790b06b-27a6-4323-ba29-eb69ce1dabd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:54:03.478236 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790b06b-27a6-4323-ba29-eb69ce1dabd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:54:03.479857 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790b06b-27a6-4323-ba29-eb69ce1dabd9 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:54:03.492597 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1464/2901] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:54:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:54:05.060807 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7113a-51b4-4d78-b7cf-7b5fed8b3075 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:54:05.061449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7113a-51b4-4d78-b7cf-7b5fed8b3075 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:54:05.061449 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7113a-51b4-4d78-b7cf-7b5fed8b3075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:54:05.078837 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5db7113a-51b4-4d78-b7cf-7b5fed8b3075 None None] MFA Rules not processed for user `55abc5e9e3a24da9abae9a2c44466b8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:54:05.080029 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5db7113a-51b4-4d78-b7cf-7b5fed8b3075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:54:05.086989 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1438/2902] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:54:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:54:05.408607 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790b06b-27a6-4323-ba29-eb69ce1dabd9 service nova] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:54:05.418361 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790b06b-27a6-4323-ba29-eb69ce1dabd9 service nova] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:54:05.421887 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205ed71-8b65-464e-8486-334150a7ac90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:54:05.422520 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205ed71-8b65-464e-8486-334150a7ac90 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:54:05.422830 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205ed71-8b65-464e-8486-334150a7ac90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:54:05.423066 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205ed71-8b65-464e-8486-334150a7ac90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:54:05.424809 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b205ed71-8b65-464e-8486-334150a7ac90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:54:05.434705 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205ed71-8b65-464e-8486-334150a7ac90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:54:05.436131 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205ed71-8b65-464e-8486-334150a7ac90 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:54:05.444034 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1465/2903] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:54:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:54:06.159487 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7113a-51b4-4d78-b7cf-7b5fed8b3075 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:54:06.172776 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7113a-51b4-4d78-b7cf-7b5fed8b3075 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:54:06.178290 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd1d52-65f5-4e81-a5e8-2a2c08b9a583 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc0d7bd7e4547f4a8d90ffc2c2e1d03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:54:06.179106 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd1d52-65f5-4e81-a5e8-2a2c08b9a583 service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:54:06.179397 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd1d52-65f5-4e81-a5e8-2a2c08b9a583 service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:54:06.179813 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd1d52-65f5-4e81-a5e8-2a2c08b9a583 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:54:06.182606 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-96bd1d52-65f5-4e81-a5e8-2a2c08b9a583 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:54:06.198519 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd1d52-65f5-4e81-a5e8-2a2c08b9a583 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:54:06.200202 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd1d52-65f5-4e81-a5e8-2a2c08b9a583 service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:54:06.213624 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1439/2904] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:54:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:54:09.125069 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205ed71-8b65-464e-8486-334150a7ac90 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:54:09.134568 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205ed71-8b65-464e-8486-334150a7ac90 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:54:09.140579 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef619ccd-6ee1-4509-a165-6ff1afc7ab08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:54:09.141253 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef619ccd-6ee1-4509-a165-6ff1afc7ab08 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:54:09.141518 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef619ccd-6ee1-4509-a165-6ff1afc7ab08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:54:09.141982 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef619ccd-6ee1-4509-a165-6ff1afc7ab08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:54:09.144229 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ef619ccd-6ee1-4509-a165-6ff1afc7ab08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:54:09.156497 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef619ccd-6ee1-4509-a165-6ff1afc7ab08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:54:09.157991 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef619ccd-6ee1-4509-a165-6ff1afc7ab08 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:54:09.173574 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1466/2905] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:54:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:54:16.207740 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=15 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:54:19.162868 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=24 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:56:01.082050 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df321af-62eb-499d-a1ac-e141e9a5cc70 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:56:01.082666 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df321af-62eb-499d-a1ac-e141e9a5cc70 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:56:01.082666 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df321af-62eb-499d-a1ac-e141e9a5cc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:56:01.101852 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5df321af-62eb-499d-a1ac-e141e9a5cc70 None None] MFA Rules not processed for user `55abc5e9e3a24da9abae9a2c44466b8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 08:56:01.102087 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5df321af-62eb-499d-a1ac-e141e9a5cc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:56:01.110932 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1440/2906] 10.222.0.17 () {58 vars in 957 bytes} [Tue Jun 7 08:56:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 08:56:11.095559 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:56:19.420015 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef619ccd-6ee1-4509-a165-6ff1afc7ab08 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:56:19.434256 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef619ccd-6ee1-4509-a165-6ff1afc7ab08 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:56:19.440742 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3db584-5617-4343-99ce-564754bea6fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:56:19.441366 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3db584-5617-4343-99ce-564754bea6fa service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:56:19.441667 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3db584-5617-4343-99ce-564754bea6fa service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:56:19.441944 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3db584-5617-4343-99ce-564754bea6fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:56:19.457075 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3db584-5617-4343-99ce-564754bea6fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:56:19.458997 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3db584-5617-4343-99ce-564754bea6fa service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:56:19.468418 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1467/2907] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:56:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:56:24.493838 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df321af-62eb-499d-a1ac-e141e9a5cc70 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:56:24.506399 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df321af-62eb-499d-a1ac-e141e9a5cc70 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:56:24.511439 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a6076-f5d5-4324-85df-97eb81608bf1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc0d7bd7e4547f4a8d90ffc2c2e1d03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:56:24.512100 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a6076-f5d5-4324-85df-97eb81608bf1 service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:56:24.512385 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a6076-f5d5-4324-85df-97eb81608bf1 service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:56:24.512661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a6076-f5d5-4324-85df-97eb81608bf1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:56:24.522172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a6076-f5d5-4324-85df-97eb81608bf1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:56:24.523761 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a6076-f5d5-4324-85df-97eb81608bf1 service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:56:24.533871 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1441/2908] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:56:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:56:29.464302 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=6 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:56:34.529469 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:58:06.361443 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3db584-5617-4343-99ce-564754bea6fa service nova] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:58:06.384060 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3db584-5617-4343-99ce-564754bea6fa service nova] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:58:06.390929 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30fcf9-faa8-4036-86f0-ce5a6f192528 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee42568b059b49a198ed2c01d5efdafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:58:06.391867 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30fcf9-faa8-4036-86f0-ce5a6f192528 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:58:06.392311 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30fcf9-faa8-4036-86f0-ce5a6f192528 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:58:06.392885 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30fcf9-faa8-4036-86f0-ce5a6f192528 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:58:06.404154 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a30fcf9-faa8-4036-86f0-ce5a6f192528 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:58:06.406215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a30fcf9-faa8-4036-86f0-ce5a6f192528 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:58:06.419901 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1468/2909] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:58:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:58:16.414782 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=6 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:58:21.875363 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a6076-f5d5-4324-85df-97eb81608bf1 service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:58:21.886013 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a6076-f5d5-4324-85df-97eb81608bf1 service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:58:21.889744 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e3d54-a9d8-4cb0-af67-acfbd2a3f755 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a87f8bb06ee4a3ba7017422f4f1943e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:58:21.890396 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e3d54-a9d8-4cb0-af67-acfbd2a3f755 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:58:21.890676 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e3d54-a9d8-4cb0-af67-acfbd2a3f755 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:58:21.890975 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e3d54-a9d8-4cb0-af67-acfbd2a3f755 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:58:21.953029 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e3d54-a9d8-4cb0-af67-acfbd2a3f755 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:58:21.954504 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e3d54-a9d8-4cb0-af67-acfbd2a3f755 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:58:21.968661 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1442/2910] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:58:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:58:25.641222 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30fcf9-faa8-4036-86f0-ce5a6f192528 service nova] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:58:25.653793 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30fcf9-faa8-4036-86f0-ce5a6f192528 service nova] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:58:25.660484 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4275c-ddfa-4705-b810-cca23303087a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc0d7bd7e4547f4a8d90ffc2c2e1d03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:58:25.661208 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4275c-ddfa-4705-b810-cca23303087a service placement] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:58:25.661490 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4275c-ddfa-4705-b810-cca23303087a service placement] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:58:25.661767 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4275c-ddfa-4705-b810-cca23303087a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:58:25.664225 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ffe4275c-ddfa-4705-b810-cca23303087a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:58:25.678214 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4275c-ddfa-4705-b810-cca23303087a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:58:25.681534 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4275c-ddfa-4705-b810-cca23303087a service placement] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:58:25.692841 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1469/2911] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:58:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 08:58:29.327410 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e3d54-a9d8-4cb0-af67-acfbd2a3f755 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:58:29.329024 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c74e3d54-a9d8-4cb0-af67-acfbd2a3f755 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:58:29.345287 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e3d54-a9d8-4cb0-af67-acfbd2a3f755 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:58:29.351078 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83e32b-6b39-4328-b9be-a5688af42ddb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc0d7bd7e4547f4a8d90ffc2c2e1d03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:58:29.356134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83e32b-6b39-4328-b9be-a5688af42ddb service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:58:29.356134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83e32b-6b39-4328-b9be-a5688af42ddb service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:58:29.356134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83e32b-6b39-4328-b9be-a5688af42ddb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:58:29.356134 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6d83e32b-6b39-4328-b9be-a5688af42ddb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:58:29.368360 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83e32b-6b39-4328-b9be-a5688af42ddb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:58:29.370039 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83e32b-6b39-4328-b9be-a5688af42ddb service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:58:29.380627 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1443/2912] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 08:58:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 08:58:29.421762 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4275c-ddfa-4705-b810-cca23303087a service placement] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:58:29.433776 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4275c-ddfa-4705-b810-cca23303087a service placement] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:58:29.438348 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210a900-fbe2-42bb-9458-02284068fbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:58:29.439220 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210a900-fbe2-42bb-9458-02284068fbc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:58:29.439497 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210a900-fbe2-42bb-9458-02284068fbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:58:29.439770 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210a900-fbe2-42bb-9458-02284068fbc4 admin admin] PATH_INFO: `/v3/users/55abc5e9e3a24da9abae9a2c44466b8f` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:58:29.440973 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c210a900-fbe2-42bb-9458-02284068fbc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55abc5e9e3a24da9abae9a2c44466b8f)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:58:29.442903 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c210a900-fbe2-42bb-9458-02284068fbc4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:58:29.637360 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c210a900-fbe2-42bb-9458-02284068fbc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55abc5e9e3a24da9abae9a2c44466b8f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:29.638187 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c210a900-fbe2-42bb-9458-02284068fbc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55abc5e9e3a24da9abae9a2c44466b8f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:29.647695 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c210a900-fbe2-42bb-9458-02284068fbc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55abc5e9e3a24da9abae9a2c44466b8f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:29.648358 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c210a900-fbe2-42bb-9458-02284068fbc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55abc5e9e3a24da9abae9a2c44466b8f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:29.732132 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c210a900-fbe2-42bb-9458-02284068fbc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55abc5e9e3a24da9abae9a2c44466b8f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:29.750638 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1470/2913] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:58:29 2022] DELETE /identity/v3/users/55abc5e9e3a24da9abae9a2c44466b8f => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:58:30.031022 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83e32b-6b39-4328-b9be-a5688af42ddb service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:58:30.042654 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83e32b-6b39-4328-b9be-a5688af42ddb service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:58:30.109353 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4ecb6-59d0-4d57-8c84-af9520456f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:58:30.110295 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4ecb6-59d0-4d57-8c84-af9520456f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:58:30.110574 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4ecb6-59d0-4d57-8c84-af9520456f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:58:30.110843 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4ecb6-59d0-4d57-8c84-af9520456f54 admin admin] PATH_INFO: `/v3/projects/af50419d91204a8eb8c1dca13803aa2f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:58:30.112046 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e4ecb6-59d0-4d57-8c84-af9520456f54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af50419d91204a8eb8c1dca13803aa2f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:58:30.114005 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e4ecb6-59d0-4d57-8c84-af9520456f54 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:58:30.220282 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-52e4ecb6-59d0-4d57-8c84-af9520456f54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af50419d91204a8eb8c1dca13803aa2f', 'request_id': 'req-52e4ecb6-59d0-4d57-8c84-af9520456f54'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:30.226106 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-52e4ecb6-59d0-4d57-8c84-af9520456f54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af50419d91204a8eb8c1dca13803aa2f', 'request_id': 'req-52e4ecb6-59d0-4d57-8c84-af9520456f54'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:30.232447 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-52e4ecb6-59d0-4d57-8c84-af9520456f54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af50419d91204a8eb8c1dca13803aa2f', 'request_id': 'req-52e4ecb6-59d0-4d57-8c84-af9520456f54'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:30.332280 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-52e4ecb6-59d0-4d57-8c84-af9520456f54 admin admin] The token cache is being invalidate because project af50419d91204a8eb8c1dca13803aa2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:58:30.333027 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-52e4ecb6-59d0-4d57-8c84-af9520456f54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:30.333128 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-52e4ecb6-59d0-4d57-8c84-af9520456f54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:30.336011 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1444/2914] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:58:30 2022] DELETE /identity/v3/projects/af50419d91204a8eb8c1dca13803aa2f => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:58:30.338743 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210a900-fbe2-42bb-9458-02284068fbc4 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:58:30.340215 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c210a900-fbe2-42bb-9458-02284068fbc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:58:30.353754 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210a900-fbe2-42bb-9458-02284068fbc4 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:58:30.414023 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716a609-da62-4ebf-94a3-35c24e3fad5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:58:30.414895 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716a609-da62-4ebf-94a3-35c24e3fad5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:58:30.415175 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716a609-da62-4ebf-94a3-35c24e3fad5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:58:30.415448 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716a609-da62-4ebf-94a3-35c24e3fad5b admin admin] PATH_INFO: `/v3/users/9e78d2d52cb94e519a14aba6e8705b62` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:58:30.416616 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716a609-da62-4ebf-94a3-35c24e3fad5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e78d2d52cb94e519a14aba6e8705b62)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:58:30.419436 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716a609-da62-4ebf-94a3-35c24e3fad5b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:58:30.613484 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f716a609-da62-4ebf-94a3-35c24e3fad5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e78d2d52cb94e519a14aba6e8705b62'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:30.614246 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f716a609-da62-4ebf-94a3-35c24e3fad5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e78d2d52cb94e519a14aba6e8705b62'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:30.621949 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f716a609-da62-4ebf-94a3-35c24e3fad5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e78d2d52cb94e519a14aba6e8705b62'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:30.622753 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f716a609-da62-4ebf-94a3-35c24e3fad5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e78d2d52cb94e519a14aba6e8705b62'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:30.627588 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:58:30.716729 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f716a609-da62-4ebf-94a3-35c24e3fad5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e78d2d52cb94e519a14aba6e8705b62'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:30.734819 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1471/2915] 10.222.0.17 () {58 vars in 1223 bytes} [Tue Jun 7 08:58:30 2022] DELETE /identity/v3/users/9e78d2d52cb94e519a14aba6e8705b62 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:58:31.038334 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4ecb6-59d0-4d57-8c84-af9520456f54 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 08:58:31.040272 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-52e4ecb6-59d0-4d57-8c84-af9520456f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 08:58:31.055172 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4ecb6-59d0-4d57-8c84-af9520456f54 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 08:58:31.064395 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:58:31.117922 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa4a6c-b1fa-42e6-a0ac-654882ed2c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ce5a782a83f41fbb386c88e6d77ba55', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df1e3ab9af2d4109a302f1229baa02a8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 08:58:31.118694 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa4a6c-b1fa-42e6-a0ac-654882ed2c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 08:58:31.118977 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa4a6c-b1fa-42e6-a0ac-654882ed2c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 08:58:31.119258 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa4a6c-b1fa-42e6-a0ac-654882ed2c6f admin admin] PATH_INFO: `/v3/projects/f583ed2bde8b41cf9215aee042b822dd` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 08:58:31.120585 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa4a6c-b1fa-42e6-a0ac-654882ed2c6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f583ed2bde8b41cf9215aee042b822dd)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 08:58:31.122623 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa4a6c-b1fa-42e6-a0ac-654882ed2c6f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 08:58:31.243069 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a5fa4a6c-b1fa-42e6-a0ac-654882ed2c6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f583ed2bde8b41cf9215aee042b822dd', 'request_id': 'req-a5fa4a6c-b1fa-42e6-a0ac-654882ed2c6f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:31.248669 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a5fa4a6c-b1fa-42e6-a0ac-654882ed2c6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f583ed2bde8b41cf9215aee042b822dd', 'request_id': 'req-a5fa4a6c-b1fa-42e6-a0ac-654882ed2c6f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:31.256125 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a5fa4a6c-b1fa-42e6-a0ac-654882ed2c6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f583ed2bde8b41cf9215aee042b822dd', 'request_id': 'req-a5fa4a6c-b1fa-42e6-a0ac-654882ed2c6f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:31.339450 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a5fa4a6c-b1fa-42e6-a0ac-654882ed2c6f admin admin] The token cache is being invalidate because project f583ed2bde8b41cf9215aee042b822dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 08:58:31.340233 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a5fa4a6c-b1fa-42e6-a0ac-654882ed2c6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:31.340470 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a5fa4a6c-b1fa-42e6-a0ac-654882ed2c6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 08:58:31.342065 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1445/2916] 10.222.0.17 () {58 vars in 1232 bytes} [Tue Jun 7 08:58:31 2022] DELETE /identity/v3/projects/f583ed2bde8b41cf9215aee042b822dd => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 08:58:41.002432 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 08:58:41.336063 dv-cinder-740146-15-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 09:01:30.680837 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716a609-da62-4ebf-94a3-35c24e3fad5b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 09:01:30.682426 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f716a609-da62-4ebf-94a3-35c24e3fad5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:01:30.819943 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716a609-da62-4ebf-94a3-35c24e3fad5b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 09:01:30.877831 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e850a-9492-483e-9989-e9b502782728 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc0d7bd7e4547f4a8d90ffc2c2e1d03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afc8781e5f864d4a9374300ee9b5193b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 09:01:30.878478 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e850a-9492-483e-9989-e9b502782728 service placement] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 09:01:30.878762 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e850a-9492-483e-9989-e9b502782728 service placement] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 09:01:30.879039 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e850a-9492-483e-9989-e9b502782728 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 09:01:30.880834 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cb7e850a-9492-483e-9989-e9b502782728 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 09:01:30.892337 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e850a-9492-483e-9989-e9b502782728 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 09:01:30.893953 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e850a-9492-483e-9989-e9b502782728 service placement] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 09:01:30.904051 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1472/2917] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Jun 7 09:01:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 09:01:40.899710 dv-cinder-740146-15-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}